S3 Programmatic access with AWS-CLI

Amazon Simple Storage Service (Amazon S3) is an object storage service that provides a secure and scalable way to store and access data on the cloud. It is designed for storing any kind of data, such as text files, images, videos, backups, and more.

Why Use AWS CLI for S3?

While the AWS Management Console is beginner-friendly, the AWS CLI offers:

Automation: You can create scripts to manage and move data seamlessly.
Flexibility: Integrate AWS operations with other command-line tools and scripts.
Bulk Operations: Easily perform actions on multiple objects or buckets at once.

Getting Started

Before we dive into commands, ensure:

You've installed the AWS CLI. If not, download and install it from the official AWS documentation. https://awscli.amazonaws.com/AWSCLIV2.msi
You've set up the CLI with your credentials using aws configure. Follow my blog https://kshitijaa.hashnode.dev/iam-programmatic-access-and-aws-cli

S3 Programmatic Operations Using AWS CLI

Here are some essential operations you can perform on S3 using the AWS CLI:

List Buckets
See all your S3 buckets:
```
 aws s3 ls
```
Create a New Bucket
Remember, bucket names must be globally unique across all of AWS:
```
 aws s3 mb s3://my-new-bucket-name
```
List Objects in a Bucket
```
 aws s3 ls s3://my-bucket-name
```

Upload Files to S3

 aws s3 cp local-file.txt s3://my-bucket-name/path/

Download Files from S3

 aws s3 cp s3://my-bucket-name/path/file.txt local-file.txt

Delete an Object

 aws s3 rm s3://my-bucket-name/path/file.txt

Synchronize Directories
Sync a local directory with an S3 bucket (great for backups!):
```
 aws s3 sync local-directory/ s3://my-bucket-name/path/
```
Set Access Permissions
You can make a file publicly accessible using the --acl flag:
```
 aws s3 cp local-file.txt s3://my-bucket-name/ --acl public-read
```

Best Practices and Tips

Security: Always follow the principle of least privilege. Only grant necessary permissions to S3 buckets and objects.
Cost Management: Regularly review and clean up unused S3 objects. Use lifecycle policies to transition older data to cheaper storage classes or archive/delete them.
Versioning: Enable versioning on your S3 buckets to keep multiple versions of an object. It helps prevent accidental overwrites or deletions.
Encryption: Use Server-Side Encryption (SSE) for sensitive data. S3 can automatically encrypt objects on upload.

Task-01

a. Launch an EC2 instance using the AWS Management Console and connect to it using Secure Shell (SSH). https://kshitijaa.hashnode.dev/connecting-to-your-linux-instance-from-windows-using-putty

b. Create an S3 bucket and upload a file to it using the AWS Management Console.

Keeping Default

Creating a public subnet

We are making this bucket a Public, and keeping all other settings as Default and creating this bucket.

c. Login to EC2 Instance (refer to step a) , Download aws cli

sudo apt install awscli

d. Access your S3 bucket from here

Check aws-cli is installed or not using checking the aws version

Once you have installed the AWS CLI, open a terminal and run the command

aws configure to configure your account credentials. Enter your AWS Access Key ID and Secret Access Key. Enter your AWS Access Key ID and Secret Access Key

List s3 buckets using below command:

you can use the aws s3 cp command to copy the file from your S3 bucket to your EC2 instance and view content of file using cat command.

Wrapping Up

The AWS CLI is a powerful tool for managing S3 programmatically. Whether you're looking to automate backups, integrate with CI/CD pipelines, or simply prefer the command line, mastering S3 operations with the AWS CLI will significantly enhance your cloud computing journey.