LLM Observability

LLM observability in Bronto is built on the OpenTelemetry GenAI semantic conventions — a standard set of gen_ai.* attributes for model calls. Any application emitting these attributes on spans or log records gets first-class, searchable prompt/response, token usage, and model fields in Bronto automatically — no Bronto-specific SDK.

This documentation was verified on July 17, 2026 and uses the latest OpenTelemetry GenAI semantic conventions available on that date. GenAI instrumentation evolves rapidly: current library releases may add the latest conventions while retaining legacy output for compatibility, so the schema emitted by default can vary by library and version.

Convention version and opt-in

To avoid breaking existing dashboards and queries, instrumentation libraries are not required to switch their default output when a newer Development convention is released. Where supported, enable the current convention with this OpenTelemetry migration opt-in:

export OTEL_SEMCONV_STABILITY_OPT_IN=gen_ai_latest_experimental

Support for this opt-in is instrumentation-specific, not universal. Check the relevant language instrumentation page and your installed package version before relying on it. Some libraries already emit the current fields by default, some require a different setting, and some still emit an older schema even when this variable is present.

1. Instrument your application

Pick your language for the generic gen_ai.* mechanics — auto-instrumentation where it exists, manual spans where it doesn’t, and the experimental content-capture flags:

Python

GenAI auto-instrumentation for supported Python model clients.

Node.js

Auto-instrumentation for OpenAI, plus alternatives.

Java

Javaagent-based GenAI coverage (also covers Kotlin).

All languages

.NET, Go, Ruby, PHP, Rust, C++, Swift, Erlang/Elixir.

Using a specific framework or AWS GenAI service? These pages cover the OTel setup and the telemetry available from each integration:

Amazon Bedrock

Converse / InvokeModel via botocore instrumentation.

Amazon Bedrock AgentCore

Agent-loop traces with the Strands framework.

OpenLLMetry

Broad provider and framework instrumentation from Traceloop.

LangChain

Chain instrumentation, with optional LangSmith OTel routing.

2. Send it to Bronto

Export through an OTel Collector or directly — see Connect OpenTelemetry to Bronto or the “Direct export to Bronto” section on any language page.

3. Know what you’ll see

The most useful gen_ai.* attributes:

Attribute	What it represents	Good for
`gen_ai.provider.name`	Model provider, e.g. `openai`, `aws.bedrock`	Filtering and grouping by provider
`gen_ai.operation.name`	Operation type, e.g. `chat`	Breaking down traffic by operation
`gen_ai.request.model` / `gen_ai.response.model`	Requested / actual model served	Comparing usage by model
`gen_ai.input.messages` / `gen_ai.output.messages`	Prompt / response payload	Inspecting exact input and generated output
`gen_ai.system_instructions`	System prompt or instruction block	Auditing assistant behavior
`gen_ai.response.finish_reasons`	Array of finish reasons, e.g. `["stop"]`	Spotting truncation or unusual stops
`gen_ai.response.time_to_first_chunk`	Seconds from issuing a streaming request until the first response chunk arrives	Measuring perceived streaming latency on individual spans
`gen_ai.usage.input_tokens` / `gen_ai.usage.output_tokens`	Token counts	Cost and usage analysis
`gen_ai.usage.cache_read.input_tokens` / `gen_ai.usage.cache_creation.input_tokens`	Provider-managed prompt-cache usage	Understanding cache effectiveness and billed input
`gen_ai.usage.reasoning.output_tokens`	Reasoning tokens, when reported	Analyzing reasoning-model usage
`gen_ai.response.id`	Provider response ID	Tracing a specific completion

OpenTelemetry also defines latency metric instruments for aggregate analysis:

Metric	What it measures	When to use it
`gen_ai.client.operation.time_to_first_chunk`	Time from issuing a request until the client receives the first response chunk	Client-side histogram for streaming responses only
`gen_ai.client.operation.duration`	Total time from issuing a request until the operation completes	Streaming and non-streaming responses; this is the time-to-completion measure for non-streaming responses
`gen_ai.server.time_to_first_token`	Time from receiving a request until the model server generates its first output token	Server-side model instrumentation, when available

Instrumentation does not yet use these boundaries consistently. Traceloop and Strands releases may emit gen_ai.server.time_to_first_token from client or framework instrumentation, while Amazon Bedrock publishes a separate CloudWatch TimeToFirstToken metric for streaming APIs. Preserve the field emitted by your instrumentation and confirm its unit and measurement boundary before comparing providers.

Content capture is instrumentation-specific and can expose sensitive data. Official Python, JavaScript, and Java instrumentations generally default it off; Traceloop and Strands differ. Prefer a supported span-attribute mode such as Python OpenAI v2’s span_only so gen_ai.input.messages / gen_ai.output.messages are queryable on traces. Where content is emitted as OTel log events—or no span-attribute mode exists—emit a structured log record correlated by trace_id:

logger.info(
    "chat.completion",
    extra={
        "gen_ai.provider.name": "openai",
        "gen_ai.operation.name": "chat",
        "gen_ai.request.model": "gpt-4o-mini",
        "gen_ai.input.messages": prompt,
        "gen_ai.output.messages": response_text,
        "gen_ai.usage.input_tokens": 33,
        "gen_ai.usage.output_tokens": 74,
    },
)

Setting attributes manually on a span (no auto-instrumentation for your provider)? Same idea:

with tracer.start_as_current_span("chat gpt-4o-mini") as span:
    span.set_attribute("gen_ai.provider.name", "openai")
    span.set_attribute("gen_ai.operation.name", "chat")
    span.set_attribute("gen_ai.request.model", "gpt-4o-mini")
    span.set_attribute("gen_ai.usage.input_tokens", 33)
    span.set_attribute("gen_ai.usage.output_tokens", 74)
    span.set_attribute("gen_ai.response.finish_reasons", ["stop"])

4. Search and aggregate

Open Log Search and use field predicates to select GenAI calls. For example, select Bedrock calls that report input-token usage:

"$gen_ai.provider.name" = 'aws.bedrock'
AND "$gen_ai.usage.input_tokens" > 0

Build token-usage views with $gen_ai.provider.name as the group-by field:

Analysis	Aggregation	Numeric field	Group by
Total input-token usage	Sum	`$gen_ai.usage.input_tokens`	`$gen_ai.provider.name`
Total output-token usage	Sum	`$gen_ai.usage.output_tokens`	`$gen_ai.provider.name`
Typical tokens per call	Median	`$gen_ai.usage.input_tokens` or `$gen_ai.usage.output_tokens`	`$gen_ai.provider.name`
High-token calls	P75, P90, P95, or P99	`$gen_ai.usage.input_tokens` or `$gen_ai.usage.output_tokens`	`$gen_ai.provider.name`
Prompt-cache activity	Sum	`$gen_ai.usage.cache_read.input_tokens` or `$gen_ai.usage.cache_creation.input_tokens`	`$gen_ai.provider.name`
Reasoning-token usage	Sum	`$gen_ai.usage.reasoning.output_tokens`	`$gen_ai.provider.name`

Bronto time-series views also provide Count, Average, Min, Max, Sum, and Median; median is the 50th percentile. Dashboard percentile views provide P75, P90, P95, and P99. Use a median to compare typical token consumption and the higher percentiles to find providers or models producing unusually large responses. For streaming latency on spans, apply the same median and percentile views to $gen_ai.response.time_to_first_chunk, grouped by $gen_ai.provider.name. If your instrumentation exports metrics, chart gen_ai.client.operation.time_to_first_chunk for client-observed streaming latency, gen_ai.client.operation.duration for total completion time, or gen_ai.server.time_to_first_token for model-server TTFT. gen_ai.response.finish_reasons remains an OpenTelemetry array attribute. Bronto indexes its first element as $gen_ai.response.finish_reasons.0; .0 is not part of the semantic-convention name and is not normally added by the Collector. See Visualizations for building these views into a dashboard.

Getting Started

MCP

AI Investigations

LLM-Powered Parsing

AI Integrations

LLM Observability

Convention version and opt-in

1. Instrument your application

Python

Node.js

Java

All languages

Amazon Bedrock

Amazon Bedrock AgentCore

OpenLLMetry

LangChain

2. Send it to Bronto

3. Know what you’ll see

4. Search and aggregate

References

​Convention version and opt-in

​1. Instrument your application

Python

Node.js

Java

All languages

Amazon Bedrock

Amazon Bedrock AgentCore

OpenLLMetry

LangChain

​2. Send it to Bronto

​3. Know what you’ll see

​4. Search and aggregate

​References

Convention version and opt-in

1. Instrument your application

2. Send it to Bronto

3. Know what you’ll see

4. Search and aggregate

References