Production AI traces and conversation costs

Production traces and Conversations live alongside Evaluations. Enable the Evaluations and Metrics modules, then open Evaluations → Production traces → Collection & prices for the environment. Metadata collection starts paused.

This is telemetry from accepted OpenTelemetry spans, not a provider invoice or a census of application traffic. R06 trace sampling, delivery loss, span limits and retention apply before AI collection. The AI collector additionally allows 100–100,000 spans per day, at most 100,000 retained observations and 64 MiB per environment. The console shows today's accepted and budget-dropped AI counts. An OTLP success receipt confirms trace acceptance; it does not promise that AI collection is enabled or that an AI-specific budget retained the derived record.

Privacy and access

Metadata reads require evals.read. Collection and price changes and deletion require evals.write, a fresh passkey, CSRF protection and an audit reason. Management tokens with read scope may read /api/v1/apps/{app}/evals/traces; ingest keys cannot read these records or change the policy. Send OTLP only with a server-side secret ingest key. Public keys cannot use the trace collector.

No prompts, responses, exception messages, system instructions, tool arguments, tool results, reasoning text or arbitrary attributes are stored by default. Recognized GenAI spans use a generic name and omit span attributes, events and links in the general Metrics trace view. The AI view stores only the supported technical identifiers and usage fields. Service and release follow the existing R06 privacy contract: use static technical labels, never customer data.

An optional preview requires both the environment's content setting and captureContent: true / capture_content=True in your instrumentation. Only the explicit saaspro.ai.content.input and .output plain-text fields are considered; standard full gen_ai.input.messages / output.messages payloads are discarded. The combined UTF-8 preview is capped at 4 KiB. Only model-operation previews are accepted. Redaction of common credentials, email addresses, URLs and IPs is best-effort; it cannot guarantee removal of all personal data. Sanitize before export and never include credentials, regulated data or private reasoning.

Previews are stored separately, expire after 1–7 days (also capped by application and AI retention), and are absent from search, audit metadata, general traces, Management API responses and page source. An explicit reveal requires evals.read plus people.pii.read and is audited. Disabling content permanently hides earlier previews immediately, including after re-enabling; the worker removes them in bounded batches. A preview already revealed to an authorized browser is not remotely recalled.

Token and price contract

The supported OTel convention snapshot was reviewed on 2026-09-08. GenAI conventions are still Development and have moved to the OpenTelemetry GenAI repository. We support model operations chat, generate_content, text_completion, embeddings, and metadata-only execute_tool, invoke_agent, create_agent. Model latency covers the observed span duration, including retries represented by that span. Agent aggregate usage is not added to model token totals.

gen_ai.usage.input_tokens includes cache-read and cache-write tokens. gen_ai.usage.output_tokens includes reasoning tokens. Counts must be nonnegative safe integers up to one billion; missing fields remain unknown. An inconsistent breakdown is marked invalid and has no cost estimate.

Each reviewed price profile specifies an exact provider, response model, pricing tier, text_tokens unit, effective UTC interval, source URL, review date and USD rates per million tokens with up to six decimal places. Your exporter explicitly declares the profile, tier and unit. Use separate profiles for different context bands, geography, cache duration, batch/flex/fast tiers or negotiated rates. The same profile ID cannot have overlapping effective intervals. The event's start time must fall within the interval. Requested model aliases never substitute for a missing or different response model.

A matching estimate uses exact integer arithmetic:

uncached input = total input - cache read - cache write
USD token subtotal = (uncached input × input rate
                     + cache read × cache-read rate
                     + cache write × cache-write rate
                     + total output × output rate) / 1,000,000

Reasoning is already part of total output. It is never charged twice. Nonzero cache categories require their own rates. To price an operation, input, output, cache-read and cache-write counts must all be known; supply an explicit zero only when the provider's contract establishes it. Audio, image or video usage cannot use these text profiles. Tool fees and aggregate agent spans are not priced. The stored price snapshot never changes when profiles are edited. There is no automatic retroactive repricing or inference from the evaluation model catalog.

The UI shows the known token subtotal and priced/model coverage. Missing prices or usage remain unknown, and tool fees, taxes, credits and other charges are excluded. Review current provider documentation before entering rates: OpenAI pricing and Anthropic pricing. The latter distinguishes 5-minute and 1-hour cache writes; geographic and service tier modifiers can change either provider's rates.

Node.js with the official OpenTelemetry exporter

Install @saaspro/node, @opentelemetry/api, @opentelemetry/sdk-trace-node, @opentelemetry/sdk-trace-base, @opentelemetry/resources and @opentelemetry/exporter-trace-otlp-http. Configure the current official SDK for your runtime as in the OpenTelemetry integration. The exporter URL is https://YOUR_CONTROL_PLANE/api/v1/otlp/v1/traces, with Authorization: Bearer ${process.env.SAASPRO_SECRET_KEY}. Keep the key server-side.

import { trace, SpanStatusCode } from '@opentelemetry/api';
import { aiTraceAttributes } from '@saaspro/node/ai-traces';

const span = trace.getTracer('customer-ai').startSpan('chat');
// Perform your own application operation, then use the actual reported usage.
span.setAttributes(aiTraceAttributes({
  operation: 'chat',
  provider: 'your-provider',
  requestModel: 'your-requested-model',
  responseModel: 'actual-response-model',
  conversationId: 'conv_existing_application_id',
  usage: { input: 1000, output: 100, cacheRead: 200, cacheWrite: 0,
           reasoning: 40 },
  pricing: { profile: 'reviewed-profile-v1', tier: 'standard-global-short',
             unit: 'text_tokens' },
}));
span.setStatus({ code: SpanStatusCode.OK });
span.end();
// Flush/shutdown your OTel provider at the normal application boundary.

The helper makes no provider requests and contains no transport or credentials. openAiUsage(response.usage) maps OpenAI total/cache/reasoning counts; anthropicUsage(response.usage) adds Anthropic's separate base input and cache categories once. Both leave missing categories unknown. They do not select a price profile, assert billing tiers or estimate missing token counts.

Python

Use the official opentelemetry-sdk and OTLP HTTP exporter, with the same secret collector endpoint and headers. saaspro.ai_traces adds no provider dependency:

from opentelemetry import trace
from saaspro.ai_traces import ai_trace_attributes, anthropic_usage

with trace.get_tracer('customer-ai').start_as_current_span('chat') as span:
    usage = anthropic_usage({
        'input_tokens': 700, 'cache_read_input_tokens': 200,
        'cache_creation_input_tokens': 100, 'output_tokens': 100,
    })
    span.set_attributes(ai_trace_attributes(
        operation='chat', provider='anthropic',
        request_model='requested-model', response_model='actual-response-model',
        conversation_id='conv_existing_application_id', usage=usage,
        pricing={'profile': 'reviewed-profile-v1',
                 'tier': 'standard-global-short', 'unit': 'text_tokens'},
    ))

openai_usage provides the corresponding OpenAI mapping. Optional content is passed separately as content={'input': 'sanitized preview'} with capture_content=True; Node uses content: { input: 'sanitized preview' } and the second argument { captureContent: true }. This opt-in does not override the server setting or preview read permissions.

Conversations, people and lifecycle

gen_ai.conversation.id is an existing opaque ID supplied by your application; we do not invent a conversation or derive it from prompt content. Identifier syntax validation is not anonymization: use opaque technical values, never a customer name, email, prompt or other personal data. It groups retained model and tool spans across traces within one environment. The latest 50 matching conversations are shown; exact-ID filtering reaches another group.

Optional anonymousId / anonymous_id must be a known analytics anonymous UUID. The collector links only to an existing person with a matching analytics session in the same organization, application and environment. It creates no person, identity or permission. The linked-person control requires the People module and people.read; PII remains governed by the existing People route. A distributed trace control requires metrics.read. Person deletion clears the link.

Deleting an AI observation removes it and its preview from cost views. Retrying the accepted R06 span cannot recreate it. Deleting the underlying trace cascades to AI observations and previews, and R06 tombstones block retries. Environment and application deletion cascade. All metadata reads immediately enforce the smallest AI, trace and application retention window; physical cleanup is bounded and rotates across environments. Historical cost views therefore describe the currently retained evidence and can decrease after deletion or expiry.