Amazon S3 Make Bucket Private Again

TL;DR: Setting upwards access control of AWS S3 consists of multiple levels, each with its own unique risk of misconfiguration. We will become through the specifics of each level and identify the dangerous cases where weak ACLs can create vulnerable configurations impacting the owner of the S3-bucket and/or through third party assets used by a lot of companies. We also show how to do information technology properly and how to monitor for these sorts of issues.

A simplified version of this write-up is available on the Detectify weblog.

Quick background

Amazon Web Services (AWS) provides a service called Simple Storage Service (S3) which exposes a storage container interface. The storage container is called a "bucket" and the files inside the bucket are called "objects". S3 provides an unlimited storage for each bucket and owners tin can utilize them to serve files. Files tin exist served either privately (via signed URLs) or publicly via an appropriately configured ACL (Access Control List) or ACP (Access Control Policy).

AWS also provides a (CDN) service called CloudFront which is often configured to apace serve S3 hosted files/objects from an optimized CloudFront server every bit close as possible to the user who is requesting the file.

Introduction

Recently, a few blog posts accept mentioned scenarios where the misconfiguration of a S3 saucepan may expose sensitive data as well as explaining that the S3 access command lists (ACL) are quite unlike to the regular user permission setup in AWS which is called Identify Access Direction (IAM).

However, we decided to approach this from a different angle. By identifying a number of different misconfigurations we discovered that we could suddenly command, monitor and interruption loftier cease websites due to weak configurations of the bucket and object ACLs.

Disclaimer

All instances disclosed below were reported to the affected parties using responsible disclosure policies. In some of the cases, third party companies were involved and we got assist from the companies afflicted to contact the vulnerable political party.

Wedo not recommend testing whatsoever of the vulnerable scenarios below without prior approval. This is especially of import in scenarios where the only mode to place the vulnerability was to actually override files and configurations. Nosotros did, withal, place one method to detect one of the vulnerable setups without actually modifying the information. You should notwithstanding make sure you're not affecting any party that has not given you written approval.

Technical details

The different misconfigurations and the impact of each depend on the following criteria:

• Who owns the S3 saucepan
• What domain is being used to serve the files from the bucket
• What type of files are inside the saucepan

We volition try to go through all different cases beneath and explain when they tin can be created with a vulnerable misconfiguration.

Identification of buckets

To start off, we need to exist able to identify buckets owned by or used past the visitor. Nosotros demand the specific bucket's proper name to make signed requests to the saucepan.

Identifying a bucket depends on the setup and also how the bucket is being reached: The request can go directly to S3, to CloudFront (or any other CDN proxy serving files from the bucket), to the S3 "Static Website" option, or more.

Some methods to identify S3-buckets are:

• Look at the HTTP-response for aServer-header which saysAmazonS3.
• Await at a random URL that doesn't exist and see if it gives you lot a S3-404, either with "Static Website enabled" or not, containingAccess Denied orNoSuchKey:
  
• The DNS-entry of the domain might reveal the bucket-name direct if the host points directly to S3.
• Endeavour accessing the root-URL. If alphabetize-list is enabled (public READ on the Bucket ACL) you will be able to run across the saucepan-name defined in<Proper noun>-chemical element.

We have identified multiple ways to make an S3-bucket actually reveal itself independent of proxies in front of it. Nosotros have notified AWS about these methods and chosen not mention them above.

If you lot do discover a domain that is pointing to a saucepan, merely cannot get the bucket proper noun, try the actual fully qualified domain proper name (FQDN) as the bucket name, this is a common setup, having the bucket named as the domain that is pointing to it.

If this doesn't work, try to:

• Google the domain and see if any history of information technology exposes the bucket name.
• Wait at response headers of objects in the saucepan to see if they have meta data that reveals the saucepan proper name.
• Look at the content and meet if it refers to whatsoever bucket. We've seen instances where avails are tagged with the saucepan name and a appointment when they were deployed.
• Brute-strength. Exist nice here, don't shoot thousands of requests confronting S3 only to find a bucket. Endeavor be clever depending on the name of the domain pointing to information technology and the actual reason why the bucket exists. If the saucepan contains sound files for ACME on the domainmedia.summit.edu, trymedia.height.edu,peak-edu-media,acme-audio oracme-media.

If the response on$bucket.s3.amazonaws.com showsNoSuchBucket you lot know the saucepan doesn't exist. An existing bucket will either give youListBucketResult orAccessDenied.

(You might as well stumble uponAllAccessDisabled, these buckets are completely dead).

Remember, simply because a bucket is named equally the company or similar, that doesn't hateful it is owned past that company. Try find references directly from the company to the bucket to confirm it is indeed owned by the specific company.

Permission/predefined groups

First, we will explore the different options that can exist used for giving admission to a requester of a saucepan and the objects inside:

ID / emailAddress

Yous are able to give access to a single user inside AWS using either the AWS user ID or their email address. This makes sense if you desire to allow a single user to have specific access to the bucket.

AuthenticatedUsers

This is probably the most misunderstood predefined group in AWS S3's ACL. Having the ACL gear up toAuthenticatedUsers basically ways "Anyone with a valid set of AWS credentials". All AWS accounts that can sign a request properly are inside this group. The requester doesn't need to have whatsoever relation at all with the AWS account owning the bucket or the object. Remember that "authenticated" is not the same matter as "authorized".

This grant is likely the well-nigh common reason a bucket is found vulnerable in the commencement place.

AllUsers

When this grant is set, the requester doesn't fifty-fifty demand to make an authenticated asking to read or write any data, anyone can make a PUT request to modify or a GET request to download an object, depending on the policy that is configured.

Policy permissions / ACP (Admission Control Policies)

The following policy permissions tin can exist prepare on the bucket or on objects within the bucket.

The ACPs on bucket and objects control different parts of S3. AWS has a list showing exactly what each grant does. In that location are more cases not mentioned below where you tin create specific IAM policies for a bucket, called a bucket-policy. Creating a saucepan-policy has its own bug, all the same, we volition merely cover the standard setup of ACLs set on buckets and objects.

READ

This gives the ability to read the content. If this ACP is set on a bucket, the requester can list the files within the bucket. If the ACP is set on an object, the content can be retrieved by the requester.

READ volition all the same work on specific objects inside a bucket, even if Object AccessREAD is not assault the complete bucket.

With the following ACL setup within AWS S3:

We tin can nonetheless read the specific object:

$ aws s3api become-object --bucket examination-saucepan --key read.txt read.txt {     "AcceptRanges": "bytes",      "ContentType": "text/plain",      "LastModified": "Sun, 09 Jul 2017 21:xiv:15 GMT",      "ContentLength": 43,      "ETag": "\"1398e667c7ebaa95284d4efa2987c1c0\"",      "Metadata": {} }

This waysREAD can be different for each object, independently of the settings on the bucket.

READ_ACP

This permission gives the ability to read the access control list of the bucket or object. If this is enabled, you tin can identify vulnerable assets without trying to modify the content or ACP at all.

READ_ACP volition even so work on specific objects within a bucket, even if Object AdmissionREAD_ACP is not assault the complete bucket.

$ aws s3api get-object-acl --bucket test-saucepan --key read-acp.txt         {     "Possessor": {         "DisplayName": "fransrosen",          ...

This meansREAD_ACP can be dissimilar for each object, independently of the settings on the bucket.

WRITE

This permission gives the ability to write content. If the bucket has this enabled for a user or group, that party can upload, change and create new files.

WRITE will non work on specific objects inside a saucepan, if Object AccessWRITE is not assault the complete bucket:

$ aws s3api put-object --bucket examination-bucket --fundamental write.txt --body write.txt  An error occurred (AccessDenied) when calling the PutObject operation: Access Denied

However, ifWRITE is assail the bucket, all objects will obey and will not be able to decide individually if they should exist writable or non:

$ aws s3api put-object --bucket exam-bucket --cardinal write.txt --body write.txt {     "ETag": "\"1398e667c7ebaa95284d4efa2987c1c0\"" }

This means,WRITE tin can be verified on the bucket in two ways, either by uploading a random file, or by modifying an existing ane.Modifying an existing file is destructive and should not be done at all. Below nosotros will explicate a way to cheque this without doing a destructive phone call, by triggering an error in between the access control bank check and the actual modification of the file.

WRITE_ACP

This permission gives the ability to alter the permission ACL of a bucket or object.

If the saucepan has this enabled for a user or a group, that party can modify the ACL of the bucket which is extremely bad. HavingWRITE_ACP on a bucket volition completely expose it to be controlled past the party having the ACP set, meaning any content of any object can now be controlled by the political party. The assailant might not exist able to READ every object already in the bucket, but they can still fully modify the existing objects. Also, the initial possessor of the S3-bucket volition go an Access Denied in the new AWS S3-panel when the attacker is claiming ownership of it when removing the READ-access on the saucepan.

Start, no access toREAD_ACP orWRITE:

$ aws s3api become-bucket-acl --bucket test-bucket  An error occurred (AccessDenied) when calling the GetBucketAcl performance: Admission Denied  $ aws s3api put-object --bucket examination-bucket --key write-acp.txt --body write-acp.txt  An error occurred (AccessDenied) when calling the PutObject performance: Access Denied

And then nosotros try to change the bucket ACL:

$ aws s3api put-saucepan-acl --bucket test-bucket --grant-total-command emailaddress=frans@example.com && repeat "success" success

The initial owner of the bucket will now see this:
(Beingness the owner, they will still be able to modify the policy of the saucepan, but this is a weird instance anyway.)

Nosotros can now command everything:

$ aws s3api get-bucket-acl --bucket test-saucepan { ...     "Grants": [         {             "Grantee": {                 "Blazon": "CanonicalUser",                  "DisplayName": "frans",                  "ID": "..."             },              "Permission": "FULL_CONTROL"  $ aws s3api put-object --bucket examination-bucket --key write-acp.txt --body write-acp.txt {     "ETag": "\"1398e667c7ebaa95284d4efa2987c1c0\"" }

A very interesting thing is thatWRITE_ACP will really all the same piece of work on specific objects inside a bucket fifty-fifty if Object AccessWRITE_ACP is not ready on the complete bucket:

$ aws s3api put-object-acl --saucepan test-bucket --key write-acp.txt --grant-write-acp uri=http://acs.amazonaws.com/groups/global/AuthenticatedUsers && echo "success" success

Also, the opposite ofWRITE applies here, havingWRITE_ACP on the saucepan, doesn't mean yous straight haveWRITE_ACP on an object:

$ aws s3api put-object-acl --bucket test-bucket --key write-acp.txt --grant-full-control emailaddress=frans@example.com  An error occurred (AccessDenied) when calling the PutObjectAcl operation: Access Denied

However, by performing the following steps when havingWRITE_ACP on the bucket you will all the same gain full access of the content of whatsoever object, by replacing the existing object with new content:

1. Change the bucket ACL:

   $ aws s3api put-bucket-acl --saucepan examination-saucepan --grant-full-control emailaddress=frans@example.com && echo "success" success

2. Modify the object (This changes you to the owner of the object):

   $ aws s3api put-object --bucket test-saucepan --cardinal write-acp.txt --body write-acp.txt {  "ETag": "\"1398e667c7ebaa95284d4efa2987c1c0\"" }

3. Change ACP of the object over again:

   $ aws s3api put-object-acl --bucket test-bucket --cardinal write1.js --grant-total-control emailaddress=frans@example.com && echo "success" success

SinceWRITE withal needs to be set on the bucket, you cannot upgrade aWRITE_ACP on an object to give yourselfWRITE on the same object:

$ aws s3api put-object-acl --saucepan exam-bucket --key write-acp.txt --grant-write-acp uri=http://acs.amazonaws.com/groups/global/AuthenticatedUsers --grant-write uri=http://acs.amazonaws.com/groups/global/AuthenticatedUsers --grant-read-acp uri=http://acs.amazonaws.com/groups/global/AuthenticatedUsers --grant-read uri=http://acs.amazonaws.com/groups/global/AuthenticatedUsers && repeat "success" success

This will even so give you:

$ aws s3api put-object --bucket examination-bucket --fundamental write-acp.txt --torso write-acp.txt  An fault occurred (AccessDenied) when calling the PutObject functioning: Access Denied

All the same, you can still remove all ACPs on the object, making the object completely private, which will stop it being served, giving a403 Forbidden.

WRITE_ACP can unfortunately only be verified by testing writing a new ACP on a saucepan or object.Modifying the existing one is of course subversive and should not be done without approval. We have not found a non-destructive mode of testing this ACP.

FULL_CONTROL

This is the policy that combines all other policies. However,WRITE will notwithstanding not work on an object unless the bucket has it set, even if this permission is set on an object.

Vulnerable scenarios

The following scenarios are cases where the company tin can be affected.

ane. Saucepan used on a domain owned by the company

Y'all found a bucket which is served past a subdomain or domain of the visitor.

You lot should test for:

• BUCKET READ
  Listing files in the bucket. Sensitive information might be exposed.
• Saucepan READ-ACP
  Let's look at the ACP and see if nosotros tin can identify the bucket being vulnerable without actually trying anything. If we see thatAllUsers orAuthenticatedUsers hasWRITE_ACP set, we know nosotros can gain full control over the bucket, without doing annihilation else.
• Saucepan WRITE (Simulate using invalid-MD5 hack)
  If we tin can upload a new file to the saucepan. This also tells united states nosotros can overwrite any object in the bucket. Notwithstanding, if we want to avoid uploading anything, we tin can endeavor the following hack, not uploading anything but still see that we are able to do information technology:
  When making a signed PUT request to a bucket, we take the option to add aContent-MD5 telling AWS the checksum of the content existence uploaded. It turns out that this check is happening inside the following flow:
  1. Check that the user has access writing the file.
  2. Cheque that the MD5-checksum is matching the content.
  3. Upload the file.

  Since the checksum control happens afterwards we know that we have access to the file, but before actually modifying it, nosotros do not need to write to the file to know that we are able to.

  The following bash code simulates this scenario:

  # use this by: ./put-simulate.sh test-saucepan/write.txt  AWS_ACCESS_KEY_ID="***" AWS_SECRET_ACCESS_KEY="***" AWS_S3_BUCKET="$(repeat "$1" | cutting -d "/" -f1)" AWS_PATH="/$(echo "$ane" | cutting -d "/" -f2-)" date=$(appointment +"%a, %d %b %Y %T %z") acl="x-amz-acl:private" content_type='application/octet-stream'  # we create a checksum of the word "yepp", but volition upload a file with the content "nope". content_md5=$(openssl dgst -md5 -binary <(echo "yepp") | openssl enc -base64)  cord="PUT\n${content_md5}\n${content_type}\n${date}\due north${acl}\north/${AWS_S3_BUCKET}${AWS_PATH}" signature=$(repeat -en "${string}" | openssl sha1 -hmac "${AWS_SECRET_ACCESS_KEY}" -binary | base64) echo "PUT to S3 with invalid md5: ${AWS_S3_BUCKET}${AWS_PATH}" upshot=$(roll -s --insecure -X PUT --information "nope" \ -H "Host: ${AWS_S3_BUCKET}.s3.amazonaws.com" \ -H "Date: $engagement" \ -H "Content-Blazon: ${content_type}" \ -H "Content-MD5: ${content_md5}" \ -H "$acl" \ -H "Potency: AWS ${AWS_ACCESS_KEY_ID}:${signature}" \ "https://${AWS_S3_BUCKET}.s3.amazonaws.com${AWS_PATH}")  if [ "$(echo ${event} | grep 'The Content-MD5 you specified did not match what we received')" != "" ]; so   echo "SUCCESS: ${AWS_S3_BUCKET}${AWS_PATH}"   get out 0 fi echo "$result" go out one

  On a bucket we tin upload to, this will result in:

  $ ./put-simulate.sh test-bucket/write.txt PUT to S3 with invalid md5: exam-saucepan/write.txt SUCCESS: test-saucepan/write.txt

  On a saucepan nosotros cannot upload to, this will effect in:

  $ ./put-simulate.sh examination-secure-saucepan/write.txt PUT to S3 with invalid md5: examination-secure-bucket/write.txt <?xml version="one.0" encoding="UTF-8"?> <Error><Code>AccessDenied</Code><Bulletin>Admission Denied</Message>

  We volition therefore never modify the content, only confirm we tin can exercise information technology. This unfortunately merely works onWRITE on objects, not onWRITE_ACP equally far as we know.

• Saucepan WRITE-ACP
  The most dangerous one. Fully upgradable to full access of the bucket. Subversive phone call. Be careful. The only way to exercise this one properly is to outset effigy out how the bucket behaves to non break whatever electric current ACP. Remember that yous tin can still have admission toWRITE_ACP even though you do not have access toREAD_ACP.
  API-documentation reference
• OBJECT READ
  We can effort to read the content of files we are interested in found past Bucket READ.
• OBJECT WRITE
  No need to exam this one, since BUCKET WRITE decides fully. If Saucepan WRITE gives an error the object willnot be writable and if BUCKET WRITE is successful, the object willalways be writable.
  However, if the company using the bucket has an application where users can upload files, look at the implementation of how they make the actual file upload to S3. If the company is using a POST Policy upload, specifically look in the policy at theCondition Matching of the$key and theContent-type. Depending on if they usestarts-with you might be able to modify the content type to HTML/XML/SVG or similar, or change the location of the file being uploaded.
• OBJECT WRITE-ACP
  Nosotros tin can endeavor modifying the ACP of the specific object. It will non enable us to modify the content, but merely the access control of the file, giving us the ability to terminate files from working publicly.
  API-documentation reference

Possible vulnerabilities:

• Reflected XSS. If we tin do BUCKET READ we tin can list assets and might find vulnerable objects, like a vulnerable SWF served on the visitor's domain.
• Stored XSS / asset control. If nosotros tin do BUCKET WRITE or Saucepan WRITE-ACP (likewise meaning OBJECT WRITE) nosotros tin change existing content or create new content, beingness able to modify javascript/css-files or by uploading a new HTML-file.
• Denial of server. If nosotros tin modify the ACP of objects using OBJECT WRITE-ACP, we can prevent objects from loading publicly.
• Information disclosure. If nosotros can list objects nosotros might find sensitive information.
• RCE. If the bucket contains modifiable executables this can result in Remote Lawmaking Execution (RCE) depending on where the executables are being used and if/past whom they are being downloaded.

two. Assets from bucket used by the visitor

Additional Disclaimer:The avails being used past a company might not always exist owned by the company. Y'all need to exist extremely careful hither not to attack anyone other than the intended target who has given y'all permission to test.

There are projects trying to automate this, such every bit 2d Order. However, Second Lodge only checks for assets beingness referenced in the HTTP-response, files being loaded dynamically are non being checked. Below is a quick example of likewise checking for dynamically loaded assets using Headless Chrome.

First, start the headless version on port 9222:

"/Applications/Google Chrome.app/Contents/MacOS/Google Chrome" --remote-debugging-port=9222 --disable-gpu --headless

We can then use a small-scale script. (context.js is borrowed from the HAR-capturer-projection since that i properly closes tabs)

const CDP = crave('chrome-remote-interface'); const URL = crave('url').URL; const Context = require('./context');  async part log_requests(orig_url) {     const context = new Context({});      process.on('SIGTERM', function () {         context.destroy();     });      try {         const client = expect context.create();         const {Network, Page} = customer;         const ourl = new URL('http://' + orig_url);         const ohost = ourl.host;          Network.requestWillBeSent((params) => {             if (params.request.url.friction match('^data:')) {                 render;             }             const url = new URL(params.request.url);             panel.log(ohost + ':' + url.host + ':' + params.request.url);         });         wait Hope.all([Network.enable(), Page.enable()]);         await Page.navigate({url: 'http://' + orig_url});         look Page.loadEventFired();         await Folio.navigate({url: 'https://' + orig_url});         await Folio.loadEventFired();     } finally {         await context.destroy();     } } const url = process.argv.slice(two)[0]; log_requests(url);

Which will give us all avails on the page which we then can utilize to figure out if they are served from S3 or not:

You should test for:

• BUCKET READ-ACP
• Saucepan WRITE (By invalid-MD5 hack)
• BUCKET WRITE-ACP
• OBJECT WRITE-ACP

Possible vulnerabilities:

• Stored XSS / nugget control. If we can do BUCKET WRITE or BUCKET WRITE-ACP (likewise meaning OBJECT WRITE) nosotros can modify existing content or create new content, being able to change javascript/css-files or similar. This can be extremely bad depending on where the assets are being used, such every bit on login pages or on main pages.
• Deprival of server. If we can modify the ACP of objects using OBJECT WRITE-ACP, nosotros can prevent objects from loading publicly.
• RCE. If assets are modifiable executables this tin can result in Remote Code Execution (RCE) depending on where the executables are being used and if/by whom they are existence downloaded.

3. Saucepan randomly found, indications it'southward endemic by the company

This one is a bit complicated. You need to have clear testify and proof that the bucket is indeed owned by the company. Try to observe references from the company pointing to this bucket, such as references on their website, CI logs or open source code.

Y'all should exam for:

• BUCKET READ
• Saucepan READ-ACP
• BUCKET WRITE (Past invalid-MD5 hack)
• BUCKET WRITE-ACP
• OBJECT WRITE-ACP

Possible vulnerabilities:

• Stored XSS / asset control. If we tin do Bucket WRITE or BUCKET WRITE-ACP (likewise meaning OBJECT WRITE) we tin can modify existing content or create new content, being able to alter javascript/css-files. All the same, in this example we don't know where the files are being used so nosotros cannot know how large the impact is without talking with the visitor.
• Denial of server. If we tin change the ACP of objects using OBJECT WRITE-ACP, nosotros can forbid objects from loading publicly. We practise not know in this case if they are however.
• Information disclosure. If we tin can list objects we might find sensitive information. Merely practise this if y'all have confirmed that the bucket is indeed connected to the company you accept blessing from.
• RCE. If the saucepan contains modifiable executables this can event in Remote Lawmaking Execution (RCE) depending on where the executables are being used and if/by whom they are being downloaded.

Results

During this enquiry we were able to ostend we could control assets on high contour websites. We reported these issues straight and were able to get them solved rapidly. The post-obit categories of websites were affected:

• Countersign managers
• DNS/CDN providers
• File storage
• Gaming
• Audio and video streaming providers
• Health tracking

We identified vulnerable assets placed on the login pages of some companies.

In some cases, vulnerable assets were loaded using Google Tag Managing director (gtm.js) withal, they did not sandbox the third parties properly, running the third political party avails directly on the domain itself (not by sandboxing them usingwww.googletagmanager.com)

We got in touch on with some third party providers, both directly just also with help from the affected companies, apace identifying the result and solving it very fast.

How to stay safe

The following processes can prevent this event from happening:

1. Sandbox 3rd party avails. As presently as you are in need of third political party assets, through gtm.js or similar, try isolating the scripts either by using the iframe provided by Google Tag Manager or by placing them on a separate domain (not just using a subdomain). Likewise enquire your provider how they handle access control on their files, and if they are using S3 for file serving.
2. If you take your ain buckets, have a await through the saucepan ACLs to verifyWRITE andWRITE_ACP are only set on specific users, never on groups such every bitAllUsers orAuthenticatedUsers.
3. The hardest fix is to prevent any object in any bucket from havingWRITE_ACP, test yourself past doing aaws s3api put-object-acl with the appropriate settings using a restricted AWS-user against your ain objects in your buckets. You might need to update the ACL on every object to mitigate this completely.
4. Take a look and run into how you are uploading objects to S3 buckets and make sure yous ready the proper ACLs on both buckets and objects.
5. Do not apply a secret bucket name every bit a form of Security through Obscurity. Treat the bucket proper name like it is already public data.

On a final note

It's articulate after this research that this problem is widespread and hard to identify and completely solve, especially if the visitor uses a huge amount of buckets, created by different systems.WRITE_ACP is the most dangerous ane for reasons mentioned, both on buckets and objects.

An interesting detail when manually uploading files to S3 using Cyberduck, changing the access command on a file looks like this:

Pretty easy to accidentally pick the wrong i there.

Until next time.

What Detectify scans for

Detectify tests web applications for the following S3 misconfiguration vulnerabilities with a severity range between iv.4-9 on the CVSS scale:

• Amazon S3 bucket allows for full bearding access
• Amazon S3 saucepan allows for arbitrary file list
• Amazon S3 saucepan allows for arbitrary file upload and exposure
• Amazon S3 bucket allows for blind uploads
• Amazon S3 bucket allows arbitrary read/writes of objects
• Amazon S3 saucepan reveals ACP/ACL

conderarthemeder.blogspot.com

Source: https://labs.detectify.com/2017/07/13/a-deep-dive-into-aws-s3-access-controls-taking-full-control-over-your-assets/

Share this post