10

S3 allows you to authenticate requests for media via a signed URL. This URL can include an expiration time, after which the URL is no longer valid (http://docs.aws.amazon.com/AmazonS3/latest/dev/RESTAuthentication.html#RESTAuthenticationQueryStringAuth). Given that the URL signature is generated by HMAC-SHA1 (which I assume is infeasible to brute force?), what is the point of the expiration? Is it to prevent brute forcing (i.e. increase the security of the media URLs)? Or is the url secure regardless of expiration time and the expiration time is provided as a way to provide domain specific expiration (i.e. a bank statement might be given an expiration time of an hour so that a malicious user with access to a computer's network requests couldn't retrieve it later).

To summarize: Is the signature of the S3 URL secure regardless of expiration time? Or is a reasonably short expiration time required to keep media secure?

John Lucas
  • 203
  • 1
  • 2
  • 5

3 Answers3

9

It is useful in the following scenario:

You want to serve, say, video files, but only to certain users. You want to make it harder for them to share URLs.

Bob signs in to your site, selects a video. You generate and HMAC the URL with the expiry time, set to say 5 seconds. The page loads in 1 second and the embedded video player makes the request to AWS. The URL is still valid.

Bob looks at the HTML source, finds the video URL, copies it and sends it to his friend. The friend opens the URL but it has expired.

Neil McGuigan
  • 3,379
  • 1
  • 16
  • 20
7

The expiration time is there to limit the lifetime of the authorization to perform the allowed action by anyone in possession of the signed URL.

The expiration time in a URL cannot be changed without invalidating the signature, of course, since it's one of the inputs to HMAC, but the expiration time, itself does not contribute anything to the actual security of the signing algorithm, itself because:

  • even though it does add information to the message being hashed, it does not add any secret information

  • the fact that a URL has "expired" does not prevent a malicious actor from brute-forcing it, if we assumed brute-forcing were computationally feasible... because you do not need S3 to confirm that you have brute-forced my signing secret. When your signing algorithm produces the same output I did based on the same input, you win. Sure, that specific URL you had is now expired, but now you have my signing secret and can sign any URL you like, with any expiration time you like.

Please note that nothing I am saying here should be taken to imply that I believe the security of the signing algorithm is flawed -- my only point is that the expiration time has no impact on whatever level of security the algorithm would provide if the expiration time were not a component.

Also for clarity, since the expiration time is part of the signed message, the expiration time is tamper-resistant, by definition. A valid URL of mine that expired yesterday can't be tweaked to "expire tomorrow" and have its signature still be valid.

The Signature Version 2 algorithm you've mentioned is very straightforward. Conceptually, you canonicalize the request you're going to make, and then you run it through HMAC. Nothing involved in the request is secret, except for the secret key. The request itself (the input message to the HMAC function), including the expiration time, has no hidden components. It couldn't, because then S3 wouldn't be able to generate the same signature, having had no secret communication from me about the upcoming request -- which is how the signing algorithm works -- for any given request (HTTP verb, Content-MD5 header value if it will be sent, Content-Type header value if it will be sent, expiration time, Canonicalized X-Amz headers if they will be sent, and finally /${bucket}/${key}{$canonical_query_string}, all these elements concatenated together with \n in between.

S3 knows my secret (as, indeed, apparently all AWS services do, so they can validate my requests), and attempts to generate the same signature as I did. If it succeeds, the request is allowed if the user whose credentials signed the request is actually allowed to make the request.

If you are in possession of a signed URL I generated, you can go at it brute force by reproducing the message I would have signed (you already know everything you need, from the URL in order to do that), and at the point when you calculate the same Signature I gave you in my signed URL, you've cracked my secret key. For any given request and expiration time, there can only be exactly one correct signature.

Good luck with that cracking project, of course, because I will have rotated my key (and its accompanying secret) out of active status and disabled it after a few weeks or months, long before you have a meaningful chance to crack it.

But the point is that, just like every other component that goes into the message I'm HMAC-ing, you see &Expires= in the URL, and you know what value goes there, in order to reproduce the original message I signed. Expiration does nothing to complicate your task.

No, the expiration is strictly to control the valid lifetime of a signed URL I hand out, on the theory that unauthorized access to that URL takes a period of time to occur. The more sensitive the information, generally speaking, the shorter you should set the request expiration time to.


Side notes: Inclusion of the expiration time also gives the back-end servers a minor optimization, which you can prove for yourself -- expiration time is checked first, before the signature's validity is checked. There is, after all, no point in spending CPU cycles trying to verify a signature that's accompanied by an expiration time that clearly indicates it has expired. S3 appears to short-circuit this unnecessary work by returning the same "Request has expired" error when the expiration time has passed, without regard to whether or not the signature is valid. You can confirm this by manually changing the expiration time of a valid request. If you set it to the past, that invalidates the signature, but the error you get is "Request has expired." If you set it to the future, that also invalidates the signature, but the error you get indicates that the signature is invalid.


Also: Signature Version 4 is a much more complex -- and even more secure -- algorithm than V2, using 5 nested iterations of HMAC-SHA256... and there aren't 5 iterations because of some misguided notion that "more is better."

In truth, it took a while for the implications of this algorithm's design to sink in with me, and it seems like a rather brilliant one.

If you peel back the layers, it becomes apparent that AWS has designed this algorithm to internally (that is, within AWS) delegate trust according to the principle of least privilege.

The innermost HMAC iteration is a message consisting of today's date, signed with a secret composed of the string "AWS4" + my secret. That's the DateKey. It's only good for 7 days. The central security repository of IAM is the only entity that knows my secret, and my DateKey.

The next iteration signs the literal name the AWS region (e.g. us-west-2) using the output of the DateKey, to derive the DateRegionKey. IAM can then deliver this value to its subsystems within each region, such that they know all they need to know in order to validate my signatures for their region, but not globally.

Next, the AWS service name (e.g. s3) is signed with the DateRegionKey, to generate the DateRegionServiceKey. With each region, the IAM subsystems can generate this secret for each service, delivering to each service all that the service needs to know in order to authenticate my signatures for their service, within that one region -- and (again) nothing more.

Then, the string "aws4_request" is signed with the DateRegionServiceKey, to form my daily signing key. Since "aws4_request" is essentially just a string, each service can derive (rather than store) the signing key I'll be using to sign requests today, thus having only the information that it needs, and nothing more.

Finally, my daily signing key is used to sign each canonicalized request.

Do you see what they did, there?

No system except the IAM core system needs to know my actual secret. If there were an internal breach in, say, an S3 region's infrastructure (however unlikely that is) the information an attacker could obtain would not reveal my actual secret to them -- only the secret that S3 in that region was aware of. If the regional infrastructure were breached, credentials obtained would be useless in other regions, etc., on up the chain to the root.

The V4 algorithm is at the core of what appears to be a secure, globally-distributed, secret signing key delegation system, where everything, internally, is on a need-to-know basis. Like I say... it seems rather brilliant.

1

Even if SHA1 is supposed not secure HMAC-SHA1 is probably not affected. AFAIK, even HMAC-MD5 is still secure. All the possible known attacks aim to hash itself directly not to HMAC. Also, even a single collision of SHA1 is still not known and it would be still very expensive and time consuming to find it.

Regarding the S3 URL exp. time, I think, it has not been mentioned to supply somehow SHA1-HMAC security.

Just check the link: Is HMAC-MD5 considered secure for authenticating encrypted data?

smrt28
  • 875
  • 6
  • 12