To directly answer your question: it's necessary to verify the authenticity of a session token every time you receive a new connection. That doesn't necessarily mean every time you receive a request; if your system is never going to combine multiple users' requests into a single connection (which is very rare on client-to-server but relatively common on, say, load-balancer-to-server) then you can verify a connection's token only once and allow multiple requests over it during the lifetime of the session (though this is still riskier than verifying every request, because somebody might manage to cause the client to send a fraudulent request without knowing the token). Even when the risks are acceptable, if the session ends (user logs out, times out, etc.) before the connection breaks, you obviously need to stop trusting the connection.
As I implied in my comment, for a system like yours it really, REALLY depends on what you're storing in the cookie. For example, if you're storing the expiration data in the cookie value without cryptographic integrity, or worse-yet just relying on the expiration feature of browser cookies in general, that's risky because there isn't anything preventing somebody from modifying that expiry. Cookie expiry means almost nothing security-wise, just that the browser will delete the cookie at that point. Anybody who knows the name and value of that cookie can still use it, though. (Perhaps they pulled it out of a log months after the cookie expired; they can still set a cookie with that name and value.) You need to either track which cookies are valid server-side, or you need to include the expiry in the cookie and prevent it from being modified via some form of cryptographic integrity check, such as a digital signature or Message Authentication Code (MAC).
This is, incidentally, how JWTs work. A JWT has three parts. Two of them are data: the header (saying who issued the JWT and how long it's good for and so on) and the claims (saying who the bearer of that JWT is and what privileges they have, according to the issuer). The third part is the signature (which is often an HMAC, because they're usually shorter and cheaper to verify than public key signatures, although in some cases such signatures are more secure) which is based on a cryptographic hash of the other parts and prevents the data parts from being forged or modified without a cryptographic key that is kept secret. When you "verify" a JWT, you're performing a cryptographic operation to ensure that the JWT's signature is correct for the other parts, meaning it hasn't been tampered with.
You say "the cookie is typically the JWT itself", and that's true, but in this case "typically" obviously doesn't apply. If you're storing only part of the JWT - for example, everything except the signature - that's completely insecure; I can just forge a JWT for another user and send it to your server, and the server will think I'm that user who has authenticated and then been handed this truncated JWT as a cookie. Same deal if you're doing things like stripping off the JWT header for the cookie; the signature part is no longer valid, so you can't tell the difference between a legitimate cookie and one I made up (or edited to claim I'm some other user).
It sounds like what you're proposing doing is using the JWT from Cognito to authenticate / authorize the user, and then creating your own session token for subsequent use. This is basically how many single-sign-on (SSO) systems work; there's nothing wrong with the idea in general. However, it relies on you having a secure session token infrastructure.
There are, broadly speaking, two ways to do session tokens safely:
- You issue a short-ish token (usually just a cryptographically random string) to every authenticated user, and also store the table of valid tokens (and what users they map to) server-side.
- Advantage: the server always knows which tokens are valid; it can store expiry information where the user can't even see (much less edit) it and it can prematurely expire tokens (if the user logs out or requests to end other sessions).
- Advantage: no long-term secret that an attacker could steal to be able to forge valid tokens.
- Advantage: the only cryptographic operation is the random number generator that creates the tokens.
- Disadvantage: if you run a cluster of servers, they all need to be able to check, constantly, whether the token is valid. This requires either a distributed memory cache, some tricky behavior where you bypass load-balancing to lock a user to one server, or a whole lot of database queries.
- Verdict: simple, but doesn't scale well.
- You create a token that contains the session details (user identity, authorization, session expiry, etc.) and use a secret cryptographic key to sign the token and prevent the user from modifying or forging it. This is how JWTs work.
- Advantage: no server-side state at all. So long as every server knows the key needed to verify the tokens, the server doesn't need to remember anything about which sessions are active.
- Advantage: a token can be issued by one server and used by another. They don't even need to be owned by the same entity, so long as the validating server trusts the issuing server. Asymmetric (public key) cryptography allows the issuing server to keep secret the "private key" required to create valid tokens, while any server with the corresponding "public key" can verify them.
- Disadvantage: it's hard to expire a token early. There are ways to do it but they're messy and don't scale well either. Therefore, the tokens are usually short-lived, and are re-issued periodically (often via a "refresh token" of the first type, which is used rarely enough to not be a scalability problem).
- Disadvantage: the signing key is the linchpin of the whole system. If an attacker obtains it, then the attacker can forge valid tokens and the server won't know the difference. Periodic rotation of the signing key can help, but even a very short compromise (the period between the signing key becoming known to the attacker, and it ceasing to be a trusted signing key) can be devastating.
- Disadvantage: cryptography is hard to get right. Although this is a pretty simple cryptosystem, it's still got enough moving parts that there's a risk of somebody screwing it up. Also, there's a risk if, say, the cryptographic hash used to create the signature is broken, and somebody can create a different token body which produces the same hash. The work-around here is just to use a well-trusted implementation written and reviewed by people who know what they're doing, keep it up to date, and use modern cryptographic primitives (for example, the SHA2 or SHA3 family, rather than SHA or MD5).
- Verdict: usable with every kind and scale of system, but harder to get right and more likely to fail catastrophically.
Looking just at the list of advantages and disadvantages, you might think the first approach is better, but it turns out that the scaling thing is a huge issue for many systems. The hybrid approach (short-lived type-2 access tokens plus session-lifetime type-1 refresh tokens that can be used to get a new access token) works well, at the cost of some additional complexity, but still doesn't remove the need for the signing key to be kept absolutely secret and the access tokens to be verified securely.