IAM Roles in AWS

By Chris Maki | December 6, 2018

Using IAM Roles to Control Access

I don’t know about you but when I first started working in the cloud, I thought of it as an on-prem solution in AWS. By on-prem solution in AWS, I mean I thought about solutions in AWS the same way I’d solve problems in our on-prem data center. Over the next few posts, I’m going to talk about making the transition from on-prem designs to cloud-native or cloud-first designs.

The first topic I’m going to cover is credentials. More specifically, limiting access to an S3 bucket and not needing a password file for secure access. To get started, we’ll use the aws cli to create a bucket. You can find instructions here to install the cli and here to configure it.

Before we jump into the details, here’s what we are going to do:

  1. Create an S3 bucket

  2. Create a user-managed IAM policy

  3. Create an IAM role to restrict access to the above S3 bucket, via a user-managed policy.

  4. Create an S3 bucket policy, which references the IAM policy above, to limit access to objects in the bucket.

  5. Create an EC2 instance with and without a role to show you can avoid using credentials.

Create an S3 Bucket

Using the create-bucket aws s3api command, you’ll create an S3 bucket called ripcitysoftware (you can name the bucket anything you’d like).

1 2 3 4 5 6
$ aws s3api create-bucket --bucket ripcitysoftware --acl private \ --region us-west-2 --create-bucket-configuration LocationConstraint=us-west-2 { "Location": "http://ripcitysoftware.s3.amazonaws.com/" } ----

Item one complete:

  • Create an S3 bucket

Create IAM role and policy

Now that we’ve got our S3 bucket created, let’s create an IAM role to identify which services can access this bucket. Once we have a role, we’ll add a bucket police to the ripcitysoftware bucket to limit access to only this role.

You’re going to create two different IAM objects, an IAM Role and an IAM Policy. The Role is associated with a resource and the policy defines what the role can do.

When I first looked at roles in AWS I was surprised that you applied them to a things like an EC2 instance. I’ve typically thought of roles as something you apply to a User. The AWS Console makes this more obvious for you. When you create a role, the console shows you what entities you can apply a role to (see the screenshot below)

IAM Create Role

First, you will create an IAM Role, then an IAM Policy. Roles are applied to an entity. When you create a role you need to tell it what it’s trusted entity is (the trusted entity is described in the ec2-policy-document.json file below). The Policy defines which permissions the role will have, in the case we are giving the policy limited access to the ripcitysoftware S3 bucket. To make it easier to keep the names straight, add -policy to the end of the policy name and -role to the end of the role name.

  • Create Role & Role Policy Document

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
# Create the Policy Document $ cat << EOF > ec2-policy-document.json { "Version": "2012-10-17", "Statement": [{ "Action": "sts:AssumeRole", "Effect": "Allow", "Principal": { "Service": "ec2.amazonaws.com" } }] } EOF # Using the above json file, create the role $ aws iam create-role --role-name rcs-s3-crud-role \ --assume-role-policy-document file://ec2-policy-document.json
  • Create an IAM Policy

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
# Create the IAM Policy json file $ cat << EOF > rcs-crud-policy.json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "s3:PutObject", "s3:GetObject", "s3:ListBucket", "s3:DeleteObject" ], "Resource": [ "arn:aws:s3:::ripcitysoftware", "arn:aws:s3:::ripcitysoftware/*" ] } ] } EOF # Create an IAM Policy using the above file $ aws iam create-policy --policy-name rcs-crud-policy \ --policy-document file://rcs-crud-policy.json
  • Attach the policy to the role

1 2 3 4 5 6
# get your account number, you'll need it for the next too $ aws sts get-caller-identity --output text --query 'Account' 123456789012 # replace 123456789012 with your account number $ aws iam attach-role-policy --role-name rcs-s3-crud-role \ --policy-arn arn:aws:iam::123456789012:policy/rcs-crud-policy

Now we’ve completed the first three items from above:

  • Create an S3 bucket

  • Create a user-managed IAM policy

  • Create an IAM role to restrict access to the above S3 bucket, via a user-managed policy.

Create a bucket policy

For a more in depth discussion on what we are about to do, see this post.

First, we need to get the ID of the rcs-s3-crud-role role we just created, using the aws cli:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
$ aws iam get-role --role-name rcs-s3-crud-role { "Role": { "Description": "Allows EC2 instances to call AWS services on your behalf.", "AssumeRolePolicyDocument": { "Version": "2012-10-17", "Statement": [ { "Action": "sts:AssumeRole", "Effect": "Allow", "Principal": { "Service": "ec2.amazonaws.com" } } ] }, "MaxSessionDuration": 3600, "RoleId": "AROA_YOUR_ROLE_ID", <= YOUR ROLE ID "CreateDate": "2018-12-04T23:07:44Z", "RoleName": "rcs-s3-crud", "Path": "/", "Arn": "arn:aws:iam::YOUR_ACCOUNT_ID:role/rcs-s3-crud" } } $

You’ll need the RoleId from above, which in this case is AROA_YOUR_ROLE_ID. You’ll also need your AWS Account ID (which you found earlier). Next, get your AWS User Name ID:

1 2 3 4 5 6 7 8 9 10 11
$ aws iam get-user --user-name <yourUserName> { "User": { "UserName": "<yourUserName>", "PasswordLastUsed": "2018-12-04T20:32:33Z", "CreateDate": "2018-09-23T22:59:47Z", "UserId": "AIDA_YOUR_USER_ID", "Path": "/", "Arn": "arn:aws:iam::123456789012:user/youUserName" } }

From the above output, we want your UserId, in this example, the user ID is AIDA_YOUR_USER_ID.

We’ve got everything we need to create an airtight S3 bucket policy. Copy everything below from cat to the EOF and run it in your terminal, this will create a file called rcs-bucket-policy.json. Using your editor of choice (like vim or emacs) update the YOUR_XXX sections with your data you collected above:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
$ cat << EOF > rcs-bucket-policy.json { "Version": "2012-10-17", "Statement": [ { "Effect": "Deny", "Principal": "*", "Action": "s3:*", "Resource": [ "arn:aws:s3:::ripcitysoftware", "arn:aws:s3:::ripcitysoftware/*" ], "Condition": { "StringNotLike": { "aws:userId": [ "AROA_YOUR_ROLE_ID:*", "AIDA_YOUR_USER_ID", "YOUR_ACCOUNT_ID" ] } } } ] } EOF

We are almost done. Now that we have a bucket policy, we need to add it to the ripcitysoftware bucket (here’s the aws s3api documentation):

1
$ aws s3api put-bucket-policy --bucket ripcitysoftware --policy file://rcs-bucket-policy.json

Now we’ve completed the first four items from above:

  • Create an S3 bucket

  • Create a user-managed IAM policy

  • Create an IAM role to restrict access to the above S3 bucket, via a user-managed policy.

  • Create an S3 bucket policy

Testing what we’ve created

You’ve set everything up, now it’s time to test it. To start, you can try accessing the bucket you just created from you laptop, to make sure your account ID and user ID set setup correctly for the bucket. To test out our setup, create an empty file and copy it to your bucket:

1 2 3 4 5 6
$ touch empty-file $ aws s3 cp empty-file s3://ripcitysoftware upload: ./empty-file to s3://ripcitysoftware/empty-file $ aws s3 ls s3://ripcitysoftware 2018-12-05 11:18:43 0 empty-file $

Can’t I always acces my bucket?

That’s all well and good you might say but I’m the bucket owner, of course I can upload and list files. That’s not always the case, here’s an AWS post on the subject.

Don’t try this UNLESS you have access to your AWS root account, what I’m about to show you will block you from your own bucket.

For fun, I thought I’d remove my User ID from the local bucket policy file, then push the updated policy to S3. Here’s what happens when I try to list files in my bucket:

1 2 3 4
$ aws s3 ls s3://ripcitysoftware An error occurred (AccessDenied) when calling the ListObjectsV2 operation: Access Denied $

This shows that our bucket policy is truly blocking access to our bucket. To fix this, I added my user ID back to the local policy file, switched to my root account and pushed the bucket policy to AWS. Now we are back to normal.

Create an EC2 Instance

In order to create an EC2 instance, you need an AMI, for this example use the Amazon Linux 2 AMI. This AMI comes with the AWS cli already installed. To get the ID of the AMI use this command:

1 2 3 4 5 6
$ aws ec2 describe-images --owners amazon \ --filters 'Name=name,Values=amzn2-ami-hvm-2.0.????????-x86_64-gp2' \ 'Name=state,Values=available' \ --output json | jq -r '.Images | sort_by(.CreationDate) | last(.[]).ImageId' ami-01bbe152bf19d0289 $

EC2 Instance, with no role

Creating an EC2 instance from the command line can be pretty hairy, this page from AWS has all the details. I’m going to use the first example and tweak the create command a little. Before you run the command, you’ll need the following:

  • ID of the AMI you want to use, in this example it will be ami-01bbe152bf19d0289

  • The name of your SSH Key Pair

  • The name of a security group, preferably an SG that allows ssh access to your instance

Create the instance, once it’s started, ssh to the instance. Below you will try to copy a file to the s3 bucket ripcitysoftware. This will fail. Then try accessing the IAM metadata using the http://169.254.169.254/latest/meta-data/iam/ endpoint, this too will fail:

$ aws ec2 run-instances --image-id ami-01bbe152bf19d0289 --count 1 \
             --instance-type t2.nano --key-name <YOUR_KEY> \
       --security-groups <YOUR_SECURITY_GROUP>
{
  "Instances": [
      {
          "Monitoring": {
              "State": "disabled"
          },
 ...
}
# check the aws console for the public IP of your instance
$ ssh -i ~/.ssh/YOUR_KEY ec2-user@XX.XX.XX.XX
The authenticity of host 'XX.XX.XX.XX (XX.XX.XX.XX)' can	't be established.
ECDSA key fingerprint is SHA256:xxxxxxxxxxxxxxxxxxxxxxxx.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'XX.XXX.XXX.XXX' (ECDSA) to the list of known hosts.

       __|  __|_  )
       _|  (     /   Amazon Linux 2 AMI
      ___|\___|___|

https://aws.amazon.com/amazon-linux-2/
[ec2-user@ip-172-31-16-134 ~]$ touch blee && aws s3 cp blee s3://ripcitysoftware
upload failed: ./blee to s3://ripcitysoftware/blee Unable to locate credentials
[ec2-user@ip-172-31-16-134 ~]$ curl http://169.254.169.254/latest/meta-data/iam/
<?xml version="1.0" encoding="iso-8859-1"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
         "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
 <head>
  <title>404 - Not Found</title>
 </head>
 <body>
  <h1>404 - Not Found</h1>
 </body>
</html>

EC2 Instance, with role

Now we’ll add the rcs-s3-crud-role to this EC2 instance. Once the role is attached to the instance, you’ll be able to upload files to the bucket. There are a couple of steps required to attach a role to an ec2 instance:

  • create an instance profile

  • add your role to the instance profile

  • attache the instance profile to the ec2 instance

  • Create the instance profile

1 2 3 4 5 6 7 8 9 10 11 12 13 14
# make sure you are running these commands from your local computer $ aws iam create-instance-profile --instance-profile-name rcs-s3-crud-profile { "InstanceProfile": { "InstanceProfileId": "AIPAXXXXXXXXXXXXXX", "Roles": [], "CreateDate": "2018-12-06T16:37:23Z", "InstanceProfileName": "rcs-s3-crud-profile", "Path": "/", "Arn": "arn:aws:iam::123456789012:instance-profile/rcs-s3-crud-profile" } } $ aws iam add-role-to-instance-profile --role-name rcs-s3-crud-role \ --instance-profile-name rcs-s3-crud-profile
  • Associate the instance profile to the ec2 instance

# you need the InstanceId of the ec2 instance you are changing
$ aws ec2 describe-instances --filters Name=image-id,Values=ami-01bbe152bf19d0289 \
             --instance-ids  | grep InstanceId
                   "InstanceId": "i-0c3a4e72d163000a2",
$ aws ec2 associate-iam-instance-profile --instance-id i-0c3a4e72d163000a2 \
             --iam-instance-profile Name=rcs-s3-crud-profile
{
  "IamInstanceProfileAssociation": {
      "InstanceId": "i-01f0e9a2df16ceeba",
      "State": "associating",
      "AssociationId": "iip-assoc-02976fdf34f0a649d",
      "IamInstanceProfile": {
          "Id": "AIP_INSTANCE_PROFILE_ID",
          "Arn": "arn:aws:iam::123456789012:instance-profile/rcs-s3-crud-profile"
      }
  }
}
# confirm the role associated with your ec2 instances
$ aws ec2 describe-iam-instance-profile-associations
{
  "IamInstanceProfileAssociations": [
      {
          "InstanceId": "i-0c3a4e72d163000a2",
          "State": "associated",
          "AssociationId": "iip-assoc-0fe56cac03cdf5733",
          "IamInstanceProfile": {
              "Id": "AIPAIT4UXPF5A5VN4JW5A",
              "Arn": "arn:aws:iam::123456789012:instance-profile/rcs-s3-crud-role"
          }
      }
  ]
}
# now ssh back into your ec2 instance
$ ssh -i ~/.ssh/YOUR_KEY ec2-user@XX.XX.XX.XX
The authenticity of host 'XX.XX.XX.XX (XX.XX.XX.XX)' can't be established.
ECDSA key fingerprint is SHA256:xxxxxxxxxxxxxxxxxxxxxxxx.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'XX.XXX.XXX.XXX' (ECDSA) to the list of known hosts.

     __|  __|_  )
     _|  (     /   Amazon Linux 2 AMI
    ___|\___|___|

https://aws.amazon.com/amazon-linux-2/
[ec2-user@ip-172-31-31-20 ~]$ touch blee && aws s3 cp blee s3://ripcitysoftware
upload: ./blee to s3://ripcitysoftware/blee
[ec2-user@ip-172-31-31-20 ~]$ curl http://169.254.169.254/latest/meta-data/iam/security-credentials/rcs-s3-crud-role
{
"Code" : "Success",
"LastUpdated" : "2018-12-05T23:39:34Z",
"Type" : "AWS-HMAC",
"AccessKeyId" : "ASIAXXXXXXXXXXXXXX",
"SecretAccessKey" : "gqlEC8fXXXXXXXXXXXXXXX",
"Token" : "FQo......",
"Expiration" : "2018-12-06T06:14:15Z"
}[ec2-user@ip-172-31-31-20 ~]$

Wow, that was a lot to cover. At this point, you have completed the following:

  • Create an S3 bucket.

  • Create a user-managed IAM policy.

  • Create an IAM role to restrict access to the above S3 bucket, via a user-managed policy.

  • Create an S3 bucket policy to limit access to the bucket.

  • Create an EC2 instance to test the bucket policy and IAM role.

Don’t' forget to terminate your ec2 instance, so you aren’t charged for instances you are no longer using.

Let me know what you think of this post. Was it helpful? Do you have a different experience using IAM roles to avoid credentials?

Updates

  1. 3/2/19 - added trailing / to all aws meta-data curl examples, these endpoints need the trailing slash

  2. 2/15/19 - updated all formatting with move to full hugo site

comments powered by Disqus