CloudFormation: Change Sets, Drift and StackSets

Creating a stack is the easy part. The risky moments are updates (a property change that quietly replaces a database), manual console edits that make reality differ from the template, and rolling the same baseline to dozens of accounts. CloudFormation has a tool for each: change sets, drift detection and StackSets.

The safe update loop

flowchart TB
  EDIT[Edit template in Git pull request] --> LINT[cfn-lint and tests]
  LINT --> CS[Create change set<br/>aws cloudformation deploy --no-execute-changeset]
  CS --> REVIEW{Review: Add, Modify, Remove, Replacement}
  REVIEW -->|replacement of stateful resource| STOP[Stop and redesign]
  REVIEW -->|expected| EXEC[Execute change set]
  EXEC --> STACK[Stack UPDATE_COMPLETE]
  EXEC -->|failure| RB[Automatic rollback]
  STACK --> DRIFT[Scheduled drift detection]
  DRIFT -->|drift found| FIX[Update template or revert console change]

Change sets

A change set is a preview computed from your new template without touching anything.

aws cloudformation deploy --template-file template.yaml --stack-name prod-network \
  --parameter-overrides Env=prod --no-execute-changeset
aws cloudformation list-change-sets --stack-name prod-network
aws cloudformation describe-change-set --stack-name prod-network --change-set-name <name> \
  --query 'Changes[].ResourceChange.{Action:Action,Id:LogicalResourceId,Type:ResourceType,Replace:Replacement}'
aws cloudformation execute-change-set --stack-name prod-network --change-set-name <name>

How to read it:

Field	Meaning
Action: Add / Modify / Remove	What will happen to the resource
Replacement: False	Updated in place
Replacement: Conditional	Depends on the property or runtime value; check the documentation for that property
Replacement: True	The old resource is deleted and a new one created: new IDs, possible downtime, data loss for stateful resources

Update behaviour is documented per property (“Update requires: Replacement” or “No interruption”). Look for Replacement: True on databases, load balancers, EFS file systems, Elasticsearch domains and anything with data. In the console, the same information appears under Change sets. Make the review a required step in CI/CD: the pipeline creates the change set, posts a summary on the pull request, and a human approves execution.

Protect critical resources

Database:
  Type: AWS::RDS::DBInstance
  DeletionPolicy: Snapshot
  UpdateReplacePolicy: Snapshot
  Properties:
    DeletionProtection: true

• DeletionPolicy applies when the resource is removed from the template or the stack is deleted (Retain, Snapshot where supported, or Delete).
• UpdateReplacePolicy applies when an update replaces the resource and the old one is deleted. Many people forget it.
• Stack policies deny updates to named resources unless someone overrides the policy.
• Termination protection (aws cloudformation update-termination-protection --enable-termination-protection --stack-name prod-network) blocks stack deletion.

Pressing “rollback” is not a backup. If an update fails on a resource that cannot roll back, the stack can reach UPDATE_ROLLBACK_FAILED; fix the cause and run aws cloudformation continue-update-rollback, skipping resources only if you understand the impact.

Drift detection

Drift means a resource differs from what the template says, typically because someone changed a security group in the console during an incident.

aws cloudformation detect-stack-drift --stack-name prod-network
# returns StackDriftDetectionId
aws cloudformation describe-stack-drift-detection-status --stack-drift-detection-id <id>
aws cloudformation describe-stack-resource-drifts --stack-name prod-network \
  --stack-resource-drift-status-filters MODIFIED DELETED

Each drifted resource lists the property, the expected value and the actual value. Decide per item: bring the template up to date if the change was right, or re-apply the template to undo an unauthorised change (a no-op update may not correct drift for every resource, so test; sometimes you need to change the template or update the property explicitly). Limits to remember: drift detection supports many but not all resource types, covers only properties specified in the template (not defaults), and does not find resources created outside CloudFormation.

Run drift detection on a schedule (an EventBridge-triggered Lambda or pipeline job per stack) and alert on DRIFTED. Pair it with AWS Config or CloudTrail to learn who made the change. Deeper protection comes from IAM: if engineers cannot edit production resources directly, drift becomes rare.

StackSets: one template, many accounts and regions

A StackSet deploys one template as stack instances across multiple accounts and Regions. Typical uses: a baseline of CloudTrail, GuardDuty and Config, an IAM role for security tooling, or a log bucket policy, in every account of the organization.

# in the management (or delegated administrator) account
aws cloudformation create-stack-set --stack-set-name security-baseline \
  --template-body file://baseline.yaml --permission-model SERVICE_MANAGED \
  --auto-deployment Enabled=true,RetainStacksOnAccountRemoval=false \
  --capabilities CAPABILITY_NAMED_IAM --call-as DELEGATED_ADMIN

aws cloudformation create-stack-instances --stack-set-name security-baseline \
  --deployment-targets OrganizationalUnitIds=ou-ab12-cdef3456 \
  --regions ap-south-1 us-east-1 \
  --operation-preferences FailureToleranceCount=1,MaxConcurrentCount=5,RegionConcurrencyType=PARALLEL \
  --call-as DELEGATED_ADMIN

aws cloudformation list-stack-instances --stack-set-name security-baseline --call-as DELEGATED_ADMIN

Key points:

Topic	Detail
Service-managed permissions	Uses AWS Organizations; roles are created automatically in target accounts; supports automatic deployment to new accounts in an OU. Needs trusted access enabled (aws organizations enable-aws-service-access --service-principal member.org.stacksets.cloudformation.amazonaws.com)
Self-managed permissions	You create AWSCloudFormationStackSetAdministrationRole and AWSCloudFormationStackSetExecutionRole in each account; use for accounts outside an organization
Operation preferences	MaxConcurrentCount and FailureToleranceCount limit blast radius; start with one test account, then one OU, then all
Updates	Updating the StackSet updates all instances; an error in one account can fail the operation, so review the per-account status
Drift	detect-stack-set-drift checks all instances
Delegated administrator	Run StackSets from a designated security or platform account instead of the management account

Guard rails: exclude sandboxes deliberately, test templates in a single account first, and avoid parameters that must differ per account unless you supply parameter overrides at the instance level.

Keep stacks manageable

A single stack has a limit of 500 resources, and large stacks update slowly. Split by lifecycle and ownership:

Layer	Contents	Change frequency
Network	VPC, subnets, routes, endpoints	Rare
Data	RDS, DynamoDB, S3	Careful, with retention policies
Application	ECS or Lambda, load balancers, alarms	Frequent

Use nested stacks to reuse a pattern inside one deployment, and cross-stack references or SSM parameters between layers. Remember that updating an exported value in use by another stack is blocked.

Checklist

1. Every production update goes through a reviewed change set.
2. Stateful resources have DeletionPolicy, UpdateReplacePolicy and deletion protection.
3. Drift detection runs weekly with alerts.
4. Cross-account baselines use StackSets with small, staged rollouts.
5. Console edits in production are the exception, recorded and followed by a template fix.