mrvitej.in
Happy Dussehra! May good always win. Happy Diwali! Wishing you a bright year ahead. Merry Christmas! Warm wishes from Vizag. Happy New Year! Here’s to a great year ahead.

Archive · AWS

IAM Roles for EC2: No Keys on Servers

Give applications on EC2 temporary AWS permissions through instance profiles, scope policies tightly, require IMDSv2, and remove long-lived access keys.

4 min read

  • AWS IAM
  • Instance profile
  • AWS CLI 2.1
  • IMDSv2

The most common AWS security incident is a leaked access key: committed to Git, left in a script, or copied onto a server that later gets compromised. The fix is simple to describe: applications running on AWS should have no stored keys at all. An IAM role attached to the instance provides short-lived credentials that rotate automatically.

State of AWS on this date. Instance profiles have worked this way for years. IMDSv2, the session-token-based metadata service introduced in 2019, is the recommended setting. AWS CLI 2.1 and all current SDKs read role credentials automatically.

How the credentials arrive

flowchart TB
  APP[Application or aws cli on EC2] -->|1 PUT token request| IMDS[Instance metadata service 169.254.169.254]
  IMDS -->|session token| APP
  APP -->|2 GET security credentials with token| IMDS
  IMDS -->|temporary keys<br/>valid about 6 hours, rotated| APP
  APP -->|3 signed API call| S3[S3 bucket or other service]
  S3 --> IAMC{IAM evaluates role policies}
  IAMC -->|allowed| OK[Request succeeds]
  IAMC -->|not allowed| DENY[AccessDenied]
  ROLE[IAM role app-web-role] -.->|attached through instance profile| IMDS

The SDK or CLI performs steps 1 and 2 for you; no code change is needed beyond not supplying keys.

Create a role with least privilege

A role has a trust policy (who may assume it: here the EC2 service) and permission policies (what it may do).

cat > trust.json <<'EOF2'
{ "Version": "2012-10-17",
  "Statement": [{ "Effect": "Allow", "Principal": { "Service": "ec2.amazonaws.com" }, "Action": "sts:AssumeRole" }] }
EOF2

cat > policy.json <<'EOF2'
{ "Version": "2012-10-17",
  "Statement": [
    { "Effect": "Allow", "Action": ["s3:GetObject", "s3:PutObject"], "Resource": "arn:aws:s3:::acme-app-data/uploads/*" },
    { "Effect": "Allow", "Action": "s3:ListBucket", "Resource": "arn:aws:s3:::acme-app-data", "Condition": { "StringLike": { "s3:prefix": "uploads/*" } } },
    { "Effect": "Allow", "Action": ["secretsmanager:GetSecretValue"], "Resource": "arn:aws:secretsmanager:ap-south-1:111111111111:secret:app/db-*" }
  ] }
EOF2

aws iam create-role --role-name app-web-role --assume-role-policy-document file://trust.json
aws iam put-role-policy --role-name app-web-role --policy-name app-access --policy-document file://policy.json
aws iam create-instance-profile --instance-profile-name app-web-profile
aws iam add-role-to-instance-profile --instance-profile-name app-web-profile --role-name app-web-role
aws ec2 associate-iam-instance-profile --instance-id i-0abc123 --iam-instance-profile Name=app-web-profile

The console creates the instance profile for you when you make an EC2 role; the CLI shows it as a separate object. Notice how tightly the S3 policy is scoped: one bucket, one prefix, two actions.

Verify on the instance

TOKEN=$(curl -s -X PUT http://169.254.169.254/latest/api/token -H "X-aws-ec2-metadata-token-ttl-seconds: 300")
curl -s -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/iam/security-credentials/
aws sts get-caller-identity           # shows assumed-role/app-web-role/i-0abc123, with no keys configured
aws s3 cp test.txt s3://acme-app-data/uploads/test.txt
aws s3 ls s3://acme-app-data/                    # AccessDenied: outside the allowed prefix (good)

Lock down the metadata service

aws ec2 modify-instance-metadata-options --instance-id i-0abc123 --http-tokens required --http-put-response-hop-limit 1 --http-endpoint enabled

Requiring tokens (IMDSv2) blocks a common attack in which a vulnerable web application is tricked into requesting http://169.254.169.254/... and returning credentials. A hop limit of 1 also keeps containers on the host from reaching the metadata service unless you intend it. Set it in launch templates for every new instance.

Find and remove long-lived keys

aws iam generate-credential-report && aws iam get-credential-report --query Content --output text | base64 -d | cut -d, -f1,9,11,14,16
aws iam list-access-keys --user-name deploy-bot
aws iam get-access-key-last-used --access-key-id AKIA...
aws iam update-access-key --user-name deploy-bot --access-key-id AKIA... --status Inactive      # disable first, delete later

Disable before deleting so you can roll back if something breaks. Search Git history and CI variables for old keys with a secret scanner, and rotate anything that was ever exposed.

The same idea elsewhere

Where code runs Use
EC2 Instance profile (this post)
ECS tasks Task role (separate from the execution role)
EKS pods IAM Roles for Service Accounts (IRSA)
Lambda Execution role
CI outside AWS Short-lived role assumption, preferably OIDC federation, not stored keys
People Single sign-on and role assumption with MFA

Troubleshooting

  • Unable to locate credentials: no instance profile attached, or IMDS blocked (hop limit, firewall, container networking).
  • AccessDenied: read the error for the denied action and resource; add exactly that to the policy. CloudTrail shows the failing call, and the IAM policy simulator tests changes.
  • Changes seem ignored: credentials are cached by SDKs for a short time; a policy change takes effect within about a minute.
  • Review roles quarterly with IAM Access Analyzer and the “last accessed” data to remove unused permissions.