For business
What it is
A full record of every run: each step, its input and output, timings and model calls, viewable as one timeline.
Why it matters
When someone asks “why did it say that?”, the answer takes minutes, not days.
Risk without it
Without traces, every AI issue is guesswork.
For engineers
How it works
OpenTelemetry spans for every stage and every model or tool call, with the run id carried through the queues. Prompts and outputs are stored, redacted, so a run can be replayed.
Typical stack
- OpenTelemetry
- Langfuse / LangSmith
- X-Ray / Grafana Tempo
Pitfalls & trade-offs
- Redact before you trace.
- Sample cheap spans, keep every model call.
- Propagate ids across queues.
Example pattern · fictional data · not a client system