JSON Web Token 101: Understanding Common Pitfalls
Prefer a video? I gave a talk on this topic at CNCF Seattle: https://www.youtube.com/watch?v=7ybmox6KQ8c
JSON web tokens (JWTs) are prevalent in modern web applications. They are used to store authentication and information about a user. They offer a convinient way to see that a user is authenticated, but they may be overkill in some use-cases. In this blog I'll outline what JSON web tokens are and some common pitfalls to avoid.
A Real Life Comparison
Let’s start things off by talking about some physical world comparisons. If you have ever visited a bar you may have been asked for a form of identification before you could enter the bar and enjoy its facilities. Most likely you presented the bar security with your drivers license or passport. This piece of ID was created by an authority that the bar trusts such as the government or the department of motor vehicles. When this identification was created it was marked with special anti-forgery indicators that only the government that created it can easily produce. Once the bar security sees that they can trust this ID, next they will read the information written on the ID, such as your name and birthdate. Finally, they will check if this information meets their expectations, to confirm you can enter the bar. This process is actually quite similar to using a JSON web token. The ID is the JWT itself, the government organization that created the ID is similar to the JWT signer, and the bar security is the relying party that verifies a JWT before performing some action.
What is a JWT?
Yet another acronym in tech, what actually is a JWT? JWT stands for JSON web token, which is a direct name indicating the format of the token and where it is typically used.
JSON or “JavaScript Object Notation” is the format of a decoded JWT. If you’re a web developer this should be familiar as it is a standard way to send information between a server and client. JSON lets you encode the fields of an object in a way that can easily be read and decoded.
The W (which stands for web) indicates that typically JWTs are used for web applications. This isn’t a hard rule but JWTs were created with the web in mind, they are a popular choice for authentication in web applications, or any software that has a distributed architecture (ex: microservices) which cannot rely on checking state.
Token in this scenario does not refer to cryptocurrency at all. These tokens have no monetary value, they are a token in the direct sense of being an object that represents some fact about that person which holds them.
The Components of a JWT
So what does a JWT actually look like in reality? Well, it’s a long esoteric string of random looking numbers and letters like you see here. When JWTs are being sent across the network they are sent in this form. This is 3 segments of JSON information base64 encoded separated by dots. When the JWT is received at a server it will then be base64 decoded and the information will be read. One really important thing to note here is that a base64 encoding is not encryption. This information is meant to be decoded, so you should never store anything secret in a JWT!
Let’s break down these 3 different sections of the JWT and see what they look like decoded.
The first section of the JWT is the header. The header contains information about how the JWT should be verified. According to the JWT standard there are 2 fields in the header. The “alg” and the “typ”. The “alg” field indicates the type of algorithm that you should use to derive the signature of this token and validate that the content has not been tampered with. The "typ" as you might guess indicates the type of token, this can vary when using JWTs with some other standards, but for the most part this type will be a JWT.
The next section is the payload. This is the meat of the JWT and analogous to the information that is listed about you on your ID. This payload should contain an “iat” which contains a unix timestamp of when the JWT was issued. Probably the most important field is the “exp” which indicates the expiry. The party receiving this JSON web token will use this field to see if this token is still considered trustworthy. In this example we also have an email field which contains the email of the user that this JWT was issued for. There are many other standard fields you will see in the payload, and you can really add anything you want, but going through all the standards here would be pretty boring so we will move on.
Finally we have the signature of the JWT. This signature is used to validate that this JWT was created by an authority you expect. Going back to our ID example at the bar, this signature can be seen as similar to all the anti-forgery details the DMV puts on your drivers license to show that they are in fact the ones that created the driver’s license. This can be used to validate that the JWT has not been tampered with and that the information in the JWT can be trusted. It’s thanks to this signature that a JWT is stateless and does not require calling an external party for verification. You simply need to do a cryptographic signing operation on the first two parts of the token, and the result should match this signature.
What is a Signature?
There are 2 ways you can sign your JWTs to assure their validity, symmetrically and asymmetrically.
Symmetric Signing
The simplest form of JWT signing is to use a shared secret key that both the party creating the JWT and the party receiving the JWT have access to. In the case of a JWT this type of signature is done using hash-based message authentication codes. With this algorithm the base64 encoding of the header and the payload are passed to a hash based message authentication algorithm (such as SHA256) that generates a unique output. The relying party can then verify the JWT has not been tampered with by using the same algorithm and secret key to check that the resulting signature is the same upon receiving the JWT. This is a straightforward way to do JWT signing, but in the distributed system use-case I prefer the next method of asymmetric signing. It allows us to have one service be responsible for issuing tokens and keeps the secret in one place.
Asymmetric Signing
Asymmetric signing like use public-key cryptography (ex: RS256) for signing and validating tokens. Public-key cryptography is used for security a lot across the web, from encrypting your web traffic over HTTPS to signing Bitcoin transactions. Without getting into the math, which is actually interesting if you haven’t read into it before, RS256 signing look like this.
If this whole process seems complicated don’t worry, it will be handled by the JWT library you use. I just find it useful to have a high-level understanding of this to know why you should be able to trust the information in a JWT.
The JWT Lifecycle
Now we understand what a JWT actually is, let’s look at actually using one. Here is an example of what a micro-service architecture leveraging JWTs might look like.
When a client logs in an authentication service creates a JWT with information about them and signs it with a private key. This signed JWT is then returned to the client. On subsequent requests to another service the client sends the JWT along. Since the other service knows the public key of the authentication service it can validate the JWT the client sends and proceed accordingly.
JWT Validation
Let’s create some pseudocode with our steps to verify a JWT given our current knowledge.
- Separate the encoded JWT into its 3 segments.
- Decode the header and payload.
- From the header segment check which signature algorithm
 to use.
- Verify the signed hash matches the hash of the header and payload.
- Check if the JWT is expired.
- JWT is valid. Check if the user is authorized to perform their requested action based on the values in the JWT payload.
Seems pretty solid right, not so fast... we have run into our first common pitfall, signature algorithm confusion.
Signature Algorithm Confusion
You should never trust information coming from a client without additional validation, and checking a JWT is no different. The JWT spec states that the signature used to validate a token is specified in the header, which seems fine as it still means the contents could not have been modified… BUT they left a major foot-gun in the spec. “None” is a valid algorithm that can be specified in the header of the JWT. When the “none” algorithm is specified the party verifying the JWT skips all additional signature validation, meaning that the attacker can specify any information they want within the token. Is our validation logic susceptible to this? Yes, we can avoid this vulnerability by explicitly specifying the signature algorithm we expect.
Our improved validation steps are as follows (with step 3 updated):
- Separate the encoded JWT into its 3 segments.
- Decode the header and payload.
- Ignore the algorithm in the header of the JWT and proceed with an algorithm we expect, for example check for "RS256".
- Verify the signed hash matches the hash of the header and payload.
- Check if the JWT is expired.
- JWT is valid. Check if the user is authorized to perform their requested action based on the values in the JWT payload.
Secret Brute Forcing
In other cases where you may use a symmetric signing key another common pitfall to avoid is picking a weak symmetric key. As a user we have the information contained in a token, and the expected signature that will be generated using the secret key and the given algorithm. If the secret key used to sign this token is too short, then the attacker can simply use brute force to find the secret key used to sign this information. If they can find this secret key then they can create JWTs with any information they wish.
Store your JWTs Securely
An increasingly common attack vector recently has been to steal or intercept a user’s JWT directly rather than stealing their username and password. As 2FA is becoming more common, directly stealing the JWT allows the attacker access for the lifetime of the token without having to worry about getting the victim to approve a 2FA request. It’s also a great target because it’s more difficult to invalidate due to its stateless nature.
For the best user experience I usually recommend storing the JWT in a cookie in the browser. This allows for re-using the token without logging in again. This cookie is sensitive so it should have XSS protection. It must have the "HttpOnly" and "secure" flags set.
If it’s possible in your application its even better to never store the JWT at all and just handle it in memory. Of course this may affect the user experience as it means the JWT cannot be retrieved again once the user closes the page.
Protect your Secret Key at all Costs
Even worse than an individual JWT being stolen would be the secret key of the JWT issuer being stolen. If this secret key is compromised then an attacker could create tokens with any authentication information they desire and these tokens would be accepted by any service that trusted this key! If you’re handling a service that creates JWTs make sure you store the secret key securely. This should be done using a vaulting service or a hardware security module. Never check your secrets into the code repository.
You Might Not Even Need JWTs
As we have covered, JWTs are great for simple authentication and information about users in distributed systems, but that doesn’t mean they are always the solution to your authentication problems. If you can use a standard stateful session token that is simply looked up in a database that may be the ideal way to handle authentication in your scenario. It’s much easier to revoke stateful session tokens as it just means updating a row in the database. You can also easily update information associated with the token in the database. Make sure that the tradeoffs of statelessness are worth it in your scenario.