Mount AWS S3 Bucket on AWS EC2 [ AWS IAM Role based Authentication]

Mounting an S3 Bucket to Linux: A Step-by-Step Guide

Mounting an S3 Bucket to Linux: A Step-by-Step Guide

Mounting an S3 bucket to a Linux instance allows you to utilize scalable object storage for large files directly within your file system. This guide will walk you through the process, from installing necessary packages to creating and verifying the mount.

Step 1: Install FUSE Packages

First, you need to install the required FUSE packages. Open your terminal and execute the following command:

yum -y install automake fuse fuse-devel gcc-c++ git libcurl-devel libxml2-devel make openssl-devel

Step 2: Clone the S3FS-FUSE Repository

Next, clone the S3FS-FUSE application from GitHub:

git clone https://github.com/s3fs-fuse/s3fs-fuse

Step 3: Change Directory to the Cloned Repository

Navigate to the cloned directory:

cd s3fs-fuse

Step 4: Build and Install S3FS-FUSE

Run the following commands to build and install S3FS-FUSE:


./autogen.sh
./configure
make
make install

Step 5: Ensure Proper IAM Role Permissions

Make sure you have an IAM role with the appropriate permissions (put, get, and list access to S3).

Step 6: Create the Mount Point

Create a directory to serve as the mount point for your S3 bucket:

mkdir -p /var/s3fs-demo-fs

Step 7: Create the Target S3 Bucket

Create your S3 bucket using the AWS CLI:

aws s3 mb s3://s3fs-demo-bkt

Step 8: Mount the S3 Bucket

Mount the S3 bucket as a filesystem using the following command:

s3fs s3fs-demo-bkt /var/s3fs-demo-fs -o iam_role=ec2-to-s3

Step 9: Verify the Mount

To ensure the S3 bucket is mounted correctly, verify it with the df command:

df -h /var/s3fs-demo-fs

Step 10: Add an fstab Entry

For persistent mounting across reboots, add an entry to /etc/fstab:

echo "s3fs#s3fs-demo-bkt /var/s3fs-demo-fs fuse _netdev,allow_other,uid=1002,gid=1002,iam_role=ec2-to-s3,use_cache=/tmp,url=https://s3.us-east-1.amazonaws.com 0 0" >> /etc/fstab

Step 11: Create an Object in the Mounted S3 Bucket

Navigate to the mounted directory and create a new file to test the setup:


cd /var/s3fs-demo-fs
touch "Thanks-for-Watching"

By following these steps, you can successfully mount an S3 bucket to your Linux instance, enabling seamless interaction with S3 storage as if it were a local filesystem.

Comments

Post a Comment

Popular posts from this blog

AWS CLI Get Security Group ID with Name.[ wild card support] Bash Script

  Hi guys, If you are wondering How can I get the ID of an AWS security group if I know the name? Here is the solution. Get the Security ID with the wild card Name. #!/bin/bash #Author Praveen SearchString=<*YourSearchString*> # Note : wild cards supported for VPCS in `aws ec2 --output text --query 'Vpcs[*].{VpcId:VpcId}' describe-vpcs` ; do aws ec2 describe-security-groups --filter Name=vpc-id,Values=$VPCS Name=group-name,Values=$searchstring --query 'SecurityGroups[*].[GroupId]' --output text done 2. With the above solution we can automate the other tasks. eg: add tags to security groups. for VPCS in `aws ec2 --output text --query 'Vpcs[*].{VpcId:VpcId}' describe-vpcs` ; do echo " Tagging Search String Created Security Groups" aws ec2 describe-security-groups --filter Name=vpc-id,Values=$VPCS Name=group- name,Values=$searchstring --query 'SecurityGroups[*].[GroupId]' --output text | xargs - I {} aws ...
Read more

AWS CLI Get Security Group ID with Name. Python Boto3 [ wild card support] Python Boto3

 """ Author Praveen This Simple Boto Script will list the AWS  security group with name provide [ Wildcard supported] """ import boto3 AWS_REGION = "us-west-2" fullauth = boto3.session.Session(profile_name='<YourProfile>') ec2 = fullauth.client('ec2' , region_name=AWS_REGION) group_name = '*<YourSearchString>' response = ec2.describe_security_groups(     Filters=[         dict(Name='group-name', Values=[group_name])     ] ) for securityGroup in response['SecurityGroups']:    print("SG ID: {}, Name: {}".format(securityGroup['GroupId'], securityGroup['GroupName'])) #This will list Security Group Name along with Security Group ID     print("SG ID: {}".format(securityGroup['GroupId'])) # This will List Only Security Group IDS
Read more

AMI Age Calculator of Running AWS EC2 Instances and Generate CSV Report

#!/bin/bash # Initialize variables with default values ACCOUNTID="<Your AccountID>" REGION="<Your Region>" OUTPUT_CSV="$ACCOUNTID-$REGION-ami_age_report.csv" # Define the CSV file name rm -rf $OUTPUT_CSV # Parse command line options while getopts "a:b:" option; do case $option in a) ACCOUNTID=${OPTARG} ;; b) REGION=${OPTARG} ;; *) echo "usage: $0 [-a ACCOUNTID] [-b REGION]" >&2 exit 1 ;; esac done # List instances and AMI IDs in the specified region instances_json=$(aws ec2 describe-instances --region "$REGION" --query 'Reservations[*].Instances[*].[InstanceId,ImageId]' --output json) # For local # Get the current timestamp current_time=$(date -u +%s) # Initialize the CSV file with headers echo "AccountID,Region,InstanceID,AMIID,AMIAge (months)" > "$OUTPUT_CSV" # Iterate through instances and append to the CSV file for row in $(echo "$insta...
Read more