For business
What it is
The cost and duration of every run, broken down by step, client and feature.
Why it matters
You know the unit cost of a report and can price, budget and optimise with real numbers.
Risk without it
AI costs surprise you at month end, and slow steps go unnoticed until users complain.
For engineers
How it works
Token counts and provider prices are logged per call and aggregated per run, tenant and feature; latency percentiles per stage; budgets with alerts.
Typical stack
- gateway logs
- CloudWatch / Grafana
- cost dashboards
Pitfalls & trade-offs
- Count retries and embeddings in the cost.
- Watch p95, not the average.
- Cache hit rate is a cost metric.
Example pattern · fictional data · not a client system