Configuring S3 Proxying via Nginx

Proxying S3 through Nginx allows you to use your server as an intermediary between clients and the S3 storage, providing multiple benefits. You can hide direct access to S3, control caching to optimize traffic, modify headers for flexible request handling, and restrict access to content. Additionally, a proxy helps unify access to multiple services under a single domain, reduce traffic costs through local caching, and enable detailed request logging for analysis.

Installing Nginx

First, install Nginx if you don’t have it yet:

sudo apt update
sudo apt install nginx

Check Nginx is running using the command:

sudo systemctl status nginx

If it’s not, start the service and enable it to launch at system boot:

sudo systemctl start nginx
sudo systemctl enable nginx

Configuring Bucket Access

For proxying to work, your bucket must be public or have configured access for your server's IP address.

To allow access to the bucket from your server's IP, set a bucket policy using AWS CLI. First, create a file named bucket-policy.json with the following content.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": "*",
      "Action": "s3:GetObject",
      "Resource": "arn:aws:s3:::bucket_name/*",
      "Condition": {
        "IpAddress": {
          "aws:SourceIp": "server_IP_address"
        }
      }
    }
  ]
}

Specify the parameters:

• bucket_name — Your bucket name.
• server_IP_address — The external IP address of your server.

Apply the policy to the bucket:

aws s3api put-bucket-policy --bucket bucket_name --policy file://bucket-policy.json --endpoint-url https://s3.hostman.com

If you need to allow multiple IP addresses, add them as an array in the IpAddress field:

"IpAddress": {
  "aws:SourceIp": ["IP_address_1", "IP_address_2"]
}

Now, the bucket will only be accessible from the specified IP addresses.

Basic Configuration

Create a configuration file for proxying. For example, create a file s3_proxy.conf in /etc/nginx/sites-available:

sudo nano /etc/nginx/sites-available/s3_proxy.conf

Configure it as follows:

server {
    listen 80;
    server_name your_domain;

    location / {
        proxy_pass https://s3.hostman.com/bucket_name/;
        proxy_set_header Host s3.hostman.com;
        proxy_ssl_server_name on;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Authorization "";
        proxy_hide_header x-amz-id-2;
        proxy_hide_header x-amz-request-id;
        proxy_hide_header Set-Cookie;
        add_header Cache-Control "public, max-age=3600";
        expires 1h;
    }
}

Save the file and create a symbolic link in sites-enabled:

sudo ln -s /etc/nginx/sites-available/s3_proxy.conf /etc/nginx/sites-enabled/

Check the configuration for errors:

sudo nginx -t

If no errors are found, restart Nginx:

sudo systemctl restart nginx

Now, files from the bucket will be available at http://your_domain/file_name.

Setting Up an SSL Certificate for the Proxy

For increased security, configure HTTPS using Certbot. Install Certbot and the Nginx plugin:

sudo apt install certbot python3-certbot-nginx

Generate and install the SSL certificate:

sudo certbot --nginx -d your_domain

Certbot will automatically update the Nginx configuration for HTTPS. Once completed, Certbot will activate the certificate, and the proxy will start working over a secure connection.