divergent/docs
TracingPrivate beta

Send traces from Python

Send traces of OpenAI calls from Python with the OpenTelemetry GenAI instrumentation.

This guide sends traces from a Python application that uses the openai package, with OpenTelemetry’s own instrumentation. Tracing reads the model, tokens, prompt and completion of each call.

Before you start

You need an API key with the traces:write scope. Create one under Settings in the dashboard, or run div login.

Send traces

  1. Install the packages.

    pip install opentelemetry-sdk opentelemetry-exporter-otlp-proto-http opentelemetry-instrumentation-openai-v2

  2. Set the exporter’s endpoint and key, and name your service.

    export OTEL_EXPORTER_OTLP_ENDPOINT=https://api.divergentlabs.xyz/api/otlp
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer%20$DIVERGENT_API_KEY"
export OTEL_EXPORTER_OTLP_COMPRESSION=gzip
export OTEL_SERVICE_NAME=my-app

  3. Record prompts and completions on the spans. Without these two variables, the spans carry the model and tokens and no content.

    export OTEL_SEMCONV_STABILITY_OPT_IN=gen_ai_latest_experimental
export OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=SPAN_ONLY

  4. Set up the tracer provider before your application creates its OpenAI client, and shut it down before the process exits, so the last spans are sent.

    from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.instrumentation.openai_v2 import OpenAIInstrumentor

provider = TracerProvider()
provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter()))
trace.set_tracer_provider(provider)
OpenAIInstrumentor().instrument()

  5. Run your application, then list the traces.

    div traces --since 10m

Group calls into one trace

Each call is its own trace unless it runs inside a span of yours. Wrap a request or a task in a span, and set session.id and user.id on it to filter traces by them later:

tracer = trace.get_tracer("my-app")

with tracer.start_as_current_span("answer question") as span:
    span.set_attribute("session.id", session_id)
    span.set_attribute("user.id", user_id)
    client.chat.completions.create(model="gpt-4o-mini", messages=messages)
div traces --session "$SESSION_ID"

Keep content out of traces

Set OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=NO_CONTENT, or leave it unset. Spans then carry the model, tokens and timings, and no prompts or completions.

Large prompts

An export over 4 MiB is refused with payload_too_large, and the exporter drops that batch. If your prompts are long, send smaller batches:

export OTEL_BSP_MAX_EXPORT_BATCH_SIZE=128
esc