For business
What it is
A buffer between steps, so work waits in line instead of getting lost when something is busy or down.
Why it matters
Spikes are absorbed, failures are retried automatically and nothing disappears, even during a provider outage or a deploy.
Risk without it
A slow AI provider or a release turns into lost work and manual re-uploads.
For engineers
How it works
Each stage consumes from a queue and emits an event for the next. Retries use exponential backoff; messages that keep failing land in a dead-letter queue that raises an alert. Consumers are idempotent, so a retry never doubles the work.
Typical stack
- SQS + dead-letter queue
- EventBridge
- Step Functions
Pitfalls & trade-offs
- The visibility timeout must exceed the longest processing time.
- An unmonitored DLQ is just a slower way to lose data.
- Ordering is not guaranteed unless you choose FIFO.
In this example
In
→ event: meeting.ready
Out
queued · attempt 1 dead-letter: empty
Fictional sample · not a client system