Inspect production AI operations
Send supported OpenTelemetry spans to inspect model operations, agent and tool activity, latency and reported token usage. Group related activity into conversations with your application's identifiers. Collection is enabled per environment and has its own budgets, so an accepted trace does not automatically mean every derived AI observation was retained.
Keep cost estimates explainable
Configure reviewed price profiles for the exact provider, response model, tier and effective interval. Known usage can then produce a token-cost estimate with a saved price snapshot. Missing usage or prices stay unknown, and the interface shows coverage so a partial subtotal does not masquerade as the full cost of your application.
Compare changes before release
Evaluation runs pin prompts, model settings and datasets. Deterministic graders or judge models produce scores beside human review, and quality gates record the acceptance policy. Compare the evidence for a candidate change, then monitor the production behavior after it is released rather than assuming an offline score describes every live interaction.
Put it to work.
Start with one application and verify the data you send.
- Enable Evaluations and Metrics, then configure the environment's AI collection policy.
- Export supported GenAI spans with a server-side ingest key and actual provider usage fields.
- Review trace coverage, add verified price profiles where needed and compare evaluation runs for your next change.
Prompts, responses and tool arguments are not stored by default. Optional previews require explicit configuration and have separate access and retention controls. Estimated token subtotals exclude tool fees, taxes and other charges; they are not provider invoices.
Module availability and retention vary by plan. Compare plans and limits.