Happy Dussehra! May good always win. Happy Diwali! Wishing you a bright year ahead. Merry Christmas! Warm wishes from Vizag. Happy New Year! Here’s to a great year ahead.

Archive ยท AWS

SQS, SNS and EventBridge: Which to Use

Three AWS messaging services compared with a fan-out design, dead-letter queues, FIFO ordering, idempotent consumers, filtering, retries and monitoring.

3 min read

  • Amazon SQS
  • Amazon SNS
  • Amazon EventBridge
  • Dead-letter queues

Decoupling parts of a system with messages lets each part fail, scale and deploy independently. AWS offers three services that overlap enough to confuse people. The short version: SQS holds work, SNS pushes one message to many, and EventBridge routes events by their content.

A design that uses all three

flowchart TB
  S3E[AWS events and SaaS events] --> EB[EventBridge bus<br/>rules match on event content]
  APP[Order service] -->|PutEvents| EB
  EB -->|rule order.paid| SNS[SNS topic order-paid]
  EB -->|rule schedule| LAM[Lambda nightly job]
  SNS -->|filter policy region=IN| Q1[SQS billing-queue]
  SNS --> Q2[SQS email-queue]
  SNS --> MAIL[Email or SMS to operators]
  Q1 --> W1[Billing workers]
  Q2 --> W2[Email workers]
  Q1 -->|after 5 failed receives| DLQ1[Dead-letter queue]
  Q2 --> DLQ2[Dead-letter queue]
  DLQ1 --> ALARM[CloudWatch alarm on messages visible]

Comparison

SQS SNS EventBridge
Model Queue; consumers pull Pub/sub; service pushes Event bus with rules
Consumers One logical consumer group per queue Many subscribers per topic Many targets per rule
Persistence Holds messages up to 14 days No storage; retries delivery, then drops or dead-letters No storage; retries up to 24 hours, with optional dead-letter queue
Filtering None (consumer decides) Message attribute filter policies Rich content-based patterns on the event body
Ordering FIFO queues give per-group order FIFO topics (with FIFO queues) Not guaranteed
Typical use Buffer work, smooth spikes, retry Fan-out, notifications React to AWS and SaaS events, schedules, route across accounts

Rule of thumb: put an SQS queue in front of any consumer that does real work so spikes and failures do not lose messages; use SNS to feed several queues; use EventBridge when many sources and conditions decide where an event goes.

Build the fan-out

aws sqs create-queue --queue-name orders-dlq --region ap-south-1
aws sqs create-queue --queue-name billing \
  --attributes '{"VisibilityTimeout":"60","MessageRetentionPeriod":"345600","ReceiveMessageWaitTimeSeconds":"20","RedrivePolicy":"{\"deadLetterQueueArn\":\"arn:aws:sqs:ap-south-1:111122223333:orders-dlq\",\"maxReceiveCount\":\"5\"}"}'

aws sns create-topic --name order-paid
aws sns subscribe --topic-arn arn:aws:sns:ap-south-1:111122223333:order-paid \
  --protocol sqs --notification-endpoint arn:aws:sqs:ap-south-1:111122223333:billing \
  --attributes '{"RawMessageDelivery":"true","FilterPolicy":"{\"region\":[\"IN\"]}"}'

The queue also needs a policy that allows the topic to send messages (sqs:SendMessage with condition aws:SourceArn equal to the topic ARN). Without it, the subscription is confirmed but nothing arrives, a very common mistake.

An EventBridge rule that forwards matching events to the topic:

aws events put-rule --name order-paid --event-pattern '{"source":["shop.orders"],"detail-type":["OrderPaid"]}'
aws events put-targets --rule order-paid --targets Id=1,Arn=arn:aws:sns:ap-south-1:111122223333:order-paid

Settings that decide reliability

  • Visibility timeout: set it longer than the worst-case processing time (many teams use six times the Lambda timeout when SQS triggers Lambda), otherwise a message is processed twice concurrently.
  • Long polling (ReceiveMessageWaitTimeSeconds=20) reduces empty receives and cost.
  • Dead-letter queue with maxReceiveCount of 3 to 5, and an alarm on ApproximateNumberOfMessagesVisible > 0 for the DLQ. Re-drive messages after you fix the bug (the console offers a redrive action).
  • At-least-once delivery: standard queues can deliver duplicates and out of order. Make consumers idempotent: store a message or business ID, and ignore repeats, or use conditional writes in DynamoDB.
  • FIFO queues give ordered, deduplicated delivery per MessageGroupId, with a lower throughput than standard queues; use them only where order matters.
  • Message size limit is 256 KB; send a pointer to S3 for larger payloads.
  • Encrypt queues and topics with SSE (KMS) when messages carry sensitive data, and keep policies least-privilege.

Monitoring

Metric Why
ApproximateAgeOfOldestMessage Consumers falling behind: the best queue health signal
ApproximateNumberOfMessagesVisible Backlog size; scale workers on it
DLQ visible messages Poison messages, bugs
SNS NumberOfNotificationsFailed Subscriber problems
EventBridge FailedInvocations Target errors or permissions

Choosing quickly

  • Need to absorb bursts and retry โ†’ SQS.
  • Need one event to reach several systems โ†’ SNS to SQS.
  • Need routing by event content, schedules or AWS service events โ†’ EventBridge.
  • Need streaming with replay and strict ordering at high volume โ†’ consider Kinesis or MSK instead; these three are not streams.