Ansible Dynamic Inventory for AWS EC2

Static inventory files go stale in the cloud: auto scaling replaces instances, IPs change, and a forgotten host keeps running unpatched. The amazon.aws.aws_ec2 inventory plugin asks the EC2 API what exists and builds groups from tags, so the inventory is always the live infrastructure.

Versions in use on this date. ansible-core 2.15, the amazon.aws collection (installed from Galaxy or bundled in the Ansible 8 package) and boto3. Check the collection’s documentation for the minimum boto3 version it requires, because collection releases raise it from time to time.

How discovery works

flowchart TB
  CFG[inventory aws_ec2.yml<br/>regions, filters, keyed_groups] --> PLUGIN[amazon.aws.aws_ec2 plugin]
  CTRL[Controller or CI runner<br/>IAM role ec2 DescribeInstances] --> PLUGIN
  PLUGIN -->|DescribeInstances with filters| EC2[(EC2 API ap-south-1)]
  EC2 --> HOSTS[Instances and tags]
  HOSTS --> GRP[Groups - role_web, role_db, env_prod, az_ap_south_1a]
  HOSTS --> HV[Host variables - instance_id, private_ip_address, tags]
  GRP --> PB[ansible-playbook site.yml --limit role_web]
  HV --> PB
  PB -->|SSH via bastion or SSM Session Manager| TARGET[Instances in private subnets]

Set it up

python3 -m venv ~/.venvs/ansible && source ~/.venvs/ansible/bin/activate
pip install "ansible-core==2.15.*" boto3 botocore
ansible-galaxy collection install amazon.aws community.aws

inventory/aws_ec2.yml (the file name must end in aws_ec2.yml or aws_ec2.yaml):

plugin: amazon.aws.aws_ec2
regions:
  - ap-south-1
filters:
  tag:Env: prod
  instance-state-name: running
keyed_groups:
  - key: tags.Role
    prefix: role
  - key: placement.availability_zone
    prefix: az
hostnames:
  - tag:Name
  - private-ip-address
compose:
  ansible_host: private_ip_address
  ansible_user: "'ubuntu'"
cache: true
cache_plugin: ansible.builtin.jsonfile
cache_connection: /tmp/ansible-aws-cache
cache_timeout: 300

Reading it:

• filters use the EC2 API filter names, so only running production instances appear.
• keyed_groups make groups from attributes: an instance tagged Role=web joins role_web.
• hostnames pick the first available name: the Name tag, otherwise the private IP.
• compose sets connection variables; here Ansible connects to the private IP as ubuntu.
• cache avoids calling the API on every run (the default is no cache); lower the timeout if instances change often.

Run it:

ansible-inventory -i inventory/aws_ec2.yml --graph
ansible-inventory -i inventory/aws_ec2.yml --host web-1 | head -30
ansible role_web -i inventory/aws_ec2.yml -m ansible.builtin.ping
ansible-playbook -i inventory/aws_ec2.yml site.yml --limit role_web

Credentials and permissions

Run the controller (or CI runner) with an IAM role rather than access keys. The minimum policy for inventory is read-only:

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Action": ["ec2:DescribeInstances", "ec2:DescribeRegions", "ec2:DescribeTags"],
    "Resource": "*"
  }]
}

Locally, use AWS_PROFILE with an IAM Identity Center profile. Ansible will pick up the usual boto3 credential chain. To cover several accounts, define one inventory file per account and assume a role with assume_role_arn in each.

Many environments, one file layout

inventory/
  aws_ec2.yml            # production
  aws_ec2_staging.yml    # filters: tag:Env: staging
  group_vars/
    role_web.yml
    role_db.yml
    env_prod.yml

Group variables apply to the dynamic groups by name (role_web.yml), so configuration stays in Git while membership comes from AWS. A consistent tagging standard (Env, Role, Owner, Patch) is the real foundation, so enforce it with Config rules or tag policies.

Connect without opening SSH

Instances in private subnets with no inbound SSH can still be managed through AWS Systems Manager Session Manager. The SSM agent runs on Amazon Linux and recent Ubuntu AMIs; the instance role needs the AmazonSSMManagedInstanceCore policy, and the controller needs the Session Manager plugin installed.

# group_vars/role_web.yml
ansible_connection: community.aws.aws_ssm
ansible_aws_ssm_bucket_name: acme-ansible-ssm-transfer
ansible_aws_ssm_region: ap-south-1
ansible_aws_ssm_instance_id: "{{ instance_id }}"

The SSM connection plugin moves files through an S3 bucket, so create a private bucket with a lifecycle rule to delete temporary objects and grant the instances and controller access to it. Confirm in the collection documentation which collection ships the plugin in your installed version, because it has been moving between collections. The benefits: no inbound port 22, no key distribution, and every session logged in CloudTrail and optionally CloudWatch or S3.

Patterns that pay off

1. Patch by tag: ansible-playbook patch.yml --limit "env_prod:&role_web" -e serial=25% updates a quarter of web servers at a time.
2. Safe targeting: always run with --limit and --check first, because a dynamic inventory can include more hosts than you expect.
3. Combine with Auto Scaling: new instances should configure themselves (user data, golden image built by Packer and Ansible) and be found by Ansible only for later changes, not for first boot.
4. Identify drift: run --check --diff on a schedule from CI and alert on changes.
5. Cost control: the plugin makes one DescribeInstances call per region per run; use cache and narrow filters in big accounts.

Troubleshooting

Symptom	Likely cause
Failed to import the required Python library (botocore or boto3)	Install into the same Python that runs Ansible (the virtualenv)
Empty inventory	Wrong region, tag filter, or instance not running; test with aws ec2 describe-instances --filters ...
UnauthorizedOperation	Role lacks ec2:DescribeInstances
Hosts unreachable	ansible_host points to a private IP the controller cannot route to; use a VPN, bastion (ProxyCommand) or SSM
Group names look odd	Invalid characters are converted to underscores; check with ansible-inventory --graph