Transcript
How Can We Upload Large Files to AWS S3? rizwanrafiq.com/how-can-we-upload-large-files-to-aws-s3
Upload Large File to s3 from the Browser For large files, Amazon S3 might separate the file into multiple uploads to maximize the upload speed. The Amazon S3 console might time out uploading large files to s3 from the browser because of session timeouts. Instead of using the Amazon S3 console, try to upload large files to s3 from the browser using the AWS Command Line Interface (AWS CLI) or an AWS SDK. Note: If you use the Amazon S3 console, then the maximum upload size of a large file from the browser is 160GB. To upload a larger bigger than 160GB, you can use the AWS CLI, AWS SDK, or Amazon S3 REST API.
AWS S3 Upload File Through CLI Installing AWS CLI We will use AWS CLI for uploading large files. This is the best way to upload large files to s3. Download AWS CLI through this link, and after that, install this file into your system. https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html
Now, click this window link. When you click this link then this screen will appear. Now, install or update this setup.
1/5
When this setup is downloaded, install this download file in your system by default CLI setting. Don’t change this file path, and install this file on the C drive.
Command Prompt Now, open the Command prompt on your window search bar.
Write this command, this command will check if the AWS CLI has been installed or not in your C drive. aws configure
2/5
If this command, asks you for the Access key, it means your AWS CLI is working. Now, you can upload large files to s3 from the browser.
AWS Console Account Now, go back to your AWS console account and open Security Credential. You will see many options and keys of many different AWS Services, but you will create and download IAM Access and Secret Keys. copy the Access key & Secret key from the download file. Now, go back to your CMD Prompt and enter the AWS Access key & Secret key. Access key Secret Key ( Then press Enter) S3 bucket region
Now enter this command and check if your S3 bucket is showing here or not. aws s3 ls If this command is showing your S3 Buckets then good. It means now you can upload large files to s3 from the browser.
3/5
Now, go to the C drive, create a new folder here, and give them any name
You can see, I’ve created a folder with the name “upload” in C drive. Now, open this folder and paste some files that you want to upload on the S3 bucket.
Now, open your CMD (command prompt) and write the below-mentioned command. aws s3 cp /FOLDERNAME s3://S3BUCKETNAME/ –recursive –include “FILENAME” You can see my folder name is upload, S3 bucket name is 4dphd, and my files name. aws s3 cp /upload s3://4dphd/ –recursive –include “0_1 Knapsack Problem solution (English+Hindi) – YouTube”
Result You can see both files have been uploaded. You can also see these files in your S3 Bucket. Any type of large file can easily be uploaded through this AWS CLI. This is the best way to upload large files to AWS s3. 4/5
Also Read: AWS Lightsail — Pros, Cons & Best Resources to Learn
5/5
