To gather some of the highly used AWS resources information EC2 instances, S3 buckets, and IAM users with a shell script and save the result in a table format in a file, you can follow the steps below.

This script will include descriptions and fetch the necessary information using the AWS CLI.

Step 1: Launch an EC2 instance

I have created an EC2 instance with t2.micro, ubuntu configuration

---

Step 2: Install and set up AWS CLI

You can follow below document to configure AWS CLI

https://kshitijaa.hashnode.dev/iam-programmatic-access-and-aws-cli

---

Step 3: Create a shell script file aws-resources-info.sh as below

Create folder structure as below mkdir -p /home/ubuntu/script/output

Write following script at location /home/ubuntu/script/aws-resources-info.sh

#!/bin/bash

########################################################################
# Author: Kshitija Malwade
# Date: 18 July 2024
# version: v1
# Description : To gather AWS resources information (EC2 instances, S3 buckets, and IAM users) using shell scripting 
# and save the result in a file (aws_resources_info.txt) in table format 
########################################################################

# File to save the results
OUTPUT_FILE="/home/ubuntu/script/output/aws_resources_info.txt"

# Clear the output file if it exists
if [ -f $OUTPUT_FILE ]; then
    rm $OUTPUT_FILE
fi

# Fetch EC2 instances information
echo "Fetching EC2 instances information..."
EC2_INSTANCES=$(aws ec2 describe-instances --query "Reservations[*].Instances[*].[InstanceId,InstanceType,State.Name,PublicIpAddress,PrivateIpAddress,Tags[?Key=='Name'].Value|[0]]" --output text)
echo -e "EC2 Instances:\nInstanceId\tInstanceType\tState\tPublicIpAddress\tPrivateIpAddress\tName" > $OUTPUT_FILE
echo -e "$EC2_INSTANCES" >> $OUTPUT_FILE

# Fetch S3 buckets information
echo "Fetching S3 buckets information..."
S3_BUCKETS=$(aws s3api list-buckets --query "Buckets[*].[Name,CreationDate]" --output text)
echo -e "\nS3 Buckets:\nBucketName\tCreationDate" >> $OUTPUT_FILE
echo -e "$S3_BUCKETS" >> $OUTPUT_FILE

# Fetch IAM users information
echo "Fetching IAM users information..."
IAM_USERS=$(aws iam list-users --query "Users[*].[UserName,UserId,Arn,CreateDate]" --output text)
echo -e "\nIAM Users:\nUserName\tUserId\tArn\tCreateDate" >> $OUTPUT_FILE
echo -e "$IAM_USERS" >> $OUTPUT_FILE

echo "AWS resources information saved to $OUTPUT_FILE"

Code Description

let's break down the shell commands you provided in detail for Fetching EC2 instances information

Echo Command for Fetching EC2 Instances Information

This command simply prints the message "Fetching EC2 instances information..." to the terminal. It is used to inform the user that the script is about to fetch EC2 instances information.

Fetch EC2 Instances Information

EC2_INSTANCES=$(aws ec2 describe-instances --query "Reservations[*].Instances[*].[InstanceId,InstanceType,State.Name,PublicIpAddress,PrivateIpAddress,Tags[?Key=='Name'].Value|[0]]" --output text)

• EC2_INSTANCES=$(...): This syntax is used to execute a command and store its output in a variable named EC2_INSTANCES.

• aws ec2 describe-instances: This is an AWS CLI command that describes one or more of your instances.

• --query "Reservations[*].Instances[*].[InstanceId,InstanceType,State.Name,PublicIpAddress,PrivateIpAddress,Tags[?Key=='Name'].Value|[0]]": The --query parameter allows you to filter and format the output using JMESPath.

  • Reservations[*].Instances[*]: This part extracts all instances from all reservations.

  • [InstanceId,InstanceType,State.Name,PublicIpAddress,PrivateIpAddress,Tags[?Key=='Name'].Value|[0]]: This part specifies the fields you want to extract:

    • InstanceId: The ID of the instance.

    • InstanceType: The type of the instance.

    • State.Name: The current state of the instance.

    • PublicIpAddress: The public IP address of the instance.

    • PrivateIpAddress: The private IP address of the instance.

    • Tags[?Key=='Name'].Value|[0]: The value of the tag with the key Name, if it exists (returns the first match or null if not found).

• --output text: This option specifies that the output should be in plain text format, making it easier to manipulate in the script.

Write Header for EC2 Instances Information

echo -e "EC2 Instances:\nInstanceId\tInstanceType\tState\tPublicIpAddress\tPrivateIpAddress\tName" > $OUTPUT_FILE

• echo -e "EC2 Instances:\nInstanceId\tInstanceType\tState\tPublicIpAddress\tPrivateIpAddress\tName": The -e option enables interpretation of backslash escapes:

  • \n: Newline character, used to start a new line.

  • \t: Tab character, used to separate columns in the output.

• > $OUTPUT_FILE: This redirects the output to a file specified by the variable $OUTPUT_FILE. If the file already exists, it will be overwritten. The result is a header row in the output file with column names.

Append EC2 Instances Information to the File

echo -e "$EC2_INSTANCES" >> $OUTPUT_FILE

• echo -e "$EC2_INSTANCES": This prints the contents of the EC2_INSTANCES variable. The -e option is used to ensure any escape sequences in the variable are interpreted correctly (if there are any, although in this context it is not strictly necessary).

• >> $OUTPUT_FILE: This appends the output to the file specified by the variable $OUTPUT_FILE. If the file does not exist, it will be created.

• The provided shell commands perform the following actions:

  1. Print a message to the terminal indicating that the script is fetching EC2 instances information.

  2. Execute an AWS CLI command to describe EC2 instances, filter the output to include specific fields, and store the result in the EC2_INSTANCES variable.

  3. Write a header row to a file, defining the columns for EC2 instance information.

  4. Append the fetched EC2 instances information to the file.

The final output file will contain a table with details of the EC2 instances, including instance ID, type, state, public IP address, private IP address, and name (if the tag exists).

The same logic has applied for s3 and IAM just the parameters are different.

---

Step 4: Make the shell script file aws-resources-info.sh executable

sudo chmod +x aws-resources-info.sh

Step 5: Run the script and see the output

./aws-resources-info.sh

This is how the code write the output to a file. Open the file and see the following content.

---

Automate Using CRON JOB:

we can schedule this script to run at a particular time daily using CRON Job

Step 1: Creating a Cron Job

Create a cron job using the crontab -e command.

Step 2: Scheduling a Cron Job

I have scheduled a job run at 9 AM every day and added the below cron job :

0 9 * * * /home/ubuntu/script/aws-resources-info.sh

As evident from the script above, the output of the cron job is saved in the /home/ubuntu/script/output/aws_resources_info.txt file.

---

This script can be modified to fetch different AWS resources or to include additional fields as needed.

You can come up with innovative ideas as per your requirement and tag me :)

---

Thanks for spending your valuable time in learning to enhance your knowledge!😃🙏

---

Share this with your colleagues and friends! ➤🤝

---

𝗙𝗼𝗹𝗹𝗼𝘄 𝗺𝗲🔗╰┈➤

Hashnode: kshitijaa.hashnode.dev

LinkedIn: https://www.linkedin.com/in/kshitija-bartakke-malwade-39678b141/

---