> ## Documentation Index
> Fetch the complete documentation index at: https://docs.bronto.io/llms.txt
> Use this file to discover all available pages before exploring further.

# LLM Observability

> Capture, send, and search OpenTelemetry GenAI telemetry — prompts, responses, token usage, and model metadata — in Bronto.

LLM observability in Bronto is built on the [OpenTelemetry GenAI semantic conventions](https://github.com/open-telemetry/semantic-conventions-genai/blob/main/docs/gen-ai/gen-ai-spans.md) — a standard set of `gen_ai.*` attributes for model calls. Any application emitting these attributes on spans or log records gets first-class, searchable prompt/response, token usage, and model fields in Bronto automatically — no Bronto-specific SDK.

<Note>
  This documentation was verified on **July 17, 2026** and uses the latest OpenTelemetry GenAI semantic conventions available on that date. GenAI instrumentation evolves rapidly: current library releases may add the latest conventions while retaining legacy output for compatibility, so the schema emitted by default can vary by library and version.
</Note>

## Convention version and opt-in

To avoid breaking existing dashboards and queries, instrumentation libraries are not required to switch their default output when a newer Development convention is released. Where supported, enable the current convention with this OpenTelemetry migration opt-in:

```bash theme={"dark"}
export OTEL_SEMCONV_STABILITY_OPT_IN=gen_ai_latest_experimental
```

<Warning>
  Support for this opt-in is **instrumentation-specific**, not universal. Check the relevant [language instrumentation page](/opentelemetry/overview) and your installed package version before relying on it. Some libraries already emit the current fields by default, some require a different setting, and some still emit an older schema even when this variable is present.
</Warning>

## 1. Instrument your application

Pick your language for the generic `gen_ai.*` mechanics — auto-instrumentation where it exists, manual spans where it doesn't, and the experimental content-capture flags:

<CardGroup cols={2}>
  <Card title="Python" href="/opentelemetry/python#genai-semantic-conventions">
    GenAI auto-instrumentation for supported Python model clients.
  </Card>

  <Card title="Node.js" href="/opentelemetry/nodejs#genai-semantic-conventions">
    Auto-instrumentation for OpenAI, plus alternatives.
  </Card>

  <Card title="Java" href="/opentelemetry/java#genai-semantic-conventions">
    Javaagent-based GenAI coverage (also covers Kotlin).
  </Card>

  <Card title="All languages" href="/opentelemetry/overview">
    .NET, Go, Ruby, PHP, Rust, C++, Swift, Erlang/Elixir.
  </Card>
</CardGroup>

Using a specific framework or AWS GenAI service? These pages cover the OTel setup and the telemetry available from each integration:

<CardGroup cols={3}>
  <Card title="Amazon Bedrock" href="/ai-features/aws-bedrock">
    `Converse` / `InvokeModel` via botocore instrumentation.
  </Card>

  <Card title="Amazon Bedrock AgentCore" href="/ai-features/aws-agentcore">
    Agent-loop traces with the Strands framework.
  </Card>

  <Card title="OpenLLMetry" href="/integrations/openllmetry">
    Broad provider and framework instrumentation from Traceloop.
  </Card>

  <Card title="LangChain" href="/integrations/langchain">
    Chain instrumentation, with optional LangSmith OTel routing.
  </Card>
</CardGroup>

## 2. Send it to Bronto

Export through an OTel Collector or directly — see [Connect OpenTelemetry to Bronto](/agent-setup/open-telemetry) or the "Direct export to Bronto" section on any language page.

## 3. Know what you'll see

The most useful `gen_ai.*` attributes:

| Attribute                                                                           | What it represents                                                              | Good for                                                  |
| ----------------------------------------------------------------------------------- | ------------------------------------------------------------------------------- | --------------------------------------------------------- |
| `gen_ai.provider.name`                                                              | Model provider, e.g. `openai`, `aws.bedrock`                                    | Filtering and grouping by provider                        |
| `gen_ai.operation.name`                                                             | Operation type, e.g. `chat`                                                     | Breaking down traffic by operation                        |
| `gen_ai.request.model` / `gen_ai.response.model`                                    | Requested / actual model served                                                 | Comparing usage by model                                  |
| `gen_ai.input.messages` / `gen_ai.output.messages`                                  | Prompt / response payload                                                       | Inspecting exact input and generated output               |
| `gen_ai.system_instructions`                                                        | System prompt or instruction block                                              | Auditing assistant behavior                               |
| `gen_ai.response.finish_reasons`                                                    | Array of finish reasons, e.g. `["stop"]`                                        | Spotting truncation or unusual stops                      |
| `gen_ai.response.time_to_first_chunk`                                               | Seconds from issuing a streaming request until the first response chunk arrives | Measuring perceived streaming latency on individual spans |
| `gen_ai.usage.input_tokens` / `gen_ai.usage.output_tokens`                          | Token counts                                                                    | Cost and usage analysis                                   |
| `gen_ai.usage.cache_read.input_tokens` / `gen_ai.usage.cache_creation.input_tokens` | Provider-managed prompt-cache usage                                             | Understanding cache effectiveness and billed input        |
| `gen_ai.usage.reasoning.output_tokens`                                              | Reasoning tokens, when reported                                                 | Analyzing reasoning-model usage                           |
| `gen_ai.response.id`                                                                | Provider response ID                                                            | Tracing a specific completion                             |

OpenTelemetry also defines latency metric instruments for aggregate analysis:

| Metric                                        | What it measures                                                                      | When to use it                                                                                            |
| --------------------------------------------- | ------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------- |
| `gen_ai.client.operation.time_to_first_chunk` | Time from issuing a request until the client receives the first response chunk        | Client-side histogram for streaming responses only                                                        |
| `gen_ai.client.operation.duration`            | Total time from issuing a request until the operation completes                       | Streaming and non-streaming responses; this is the time-to-completion measure for non-streaming responses |
| `gen_ai.server.time_to_first_token`           | Time from receiving a request until the model server generates its first output token | Server-side model instrumentation, when available                                                         |

<Note>
  Instrumentation does not yet use these boundaries consistently. Traceloop and Strands releases may emit `gen_ai.server.time_to_first_token` from client or framework instrumentation, while Amazon Bedrock publishes a separate CloudWatch `TimeToFirstToken` metric for streaming APIs. Preserve the field emitted by your instrumentation and confirm its unit and measurement boundary before comparing providers.
</Note>

<Warning>
  Content capture is instrumentation-specific and can expose sensitive data. Official Python, JavaScript, and Java instrumentations generally default it off; Traceloop and Strands differ. Prefer a supported span-attribute mode such as Python OpenAI v2's `span_only` so `gen_ai.input.messages` / `gen_ai.output.messages` are queryable on traces. Where content is emitted as OTel log events—or no span-attribute mode exists—emit a structured **log record** correlated by `trace_id`:

  ```python theme={"dark"}
  logger.info(
      "chat.completion",
      extra={
          "gen_ai.provider.name": "openai",
          "gen_ai.operation.name": "chat",
          "gen_ai.request.model": "gpt-4o-mini",
          "gen_ai.input.messages": prompt,
          "gen_ai.output.messages": response_text,
          "gen_ai.usage.input_tokens": 33,
          "gen_ai.usage.output_tokens": 74,
      },
  )
  ```
</Warning>

Setting attributes manually on a span (no auto-instrumentation for your provider)? Same idea:

```python theme={"dark"}
with tracer.start_as_current_span("chat gpt-4o-mini") as span:
    span.set_attribute("gen_ai.provider.name", "openai")
    span.set_attribute("gen_ai.operation.name", "chat")
    span.set_attribute("gen_ai.request.model", "gpt-4o-mini")
    span.set_attribute("gen_ai.usage.input_tokens", 33)
    span.set_attribute("gen_ai.usage.output_tokens", 74)
    span.set_attribute("gen_ai.response.finish_reasons", ["stop"])
```

## 4. Search and aggregate

Open [Log Search](https://app.bronto.io/search) and use field predicates to select GenAI calls. For example, select Bedrock calls that report input-token usage:

```sql theme={"dark"}
"$gen_ai.provider.name" = 'aws.bedrock'
AND "$gen_ai.usage.input_tokens" > 0
```

Build token-usage views with `$gen_ai.provider.name` as the group-by field:

| Analysis                 | Aggregation                           | Numeric field                                                                          | Group by                |
| ------------------------ | ------------------------------------- | -------------------------------------------------------------------------------------- | ----------------------- |
| Total input-token usage  | **Sum**                               | `$gen_ai.usage.input_tokens`                                                           | `$gen_ai.provider.name` |
| Total output-token usage | **Sum**                               | `$gen_ai.usage.output_tokens`                                                          | `$gen_ai.provider.name` |
| Typical tokens per call  | **Median**                            | `$gen_ai.usage.input_tokens` or `$gen_ai.usage.output_tokens`                          | `$gen_ai.provider.name` |
| High-token calls         | **P75**, **P90**, **P95**, or **P99** | `$gen_ai.usage.input_tokens` or `$gen_ai.usage.output_tokens`                          | `$gen_ai.provider.name` |
| Prompt-cache activity    | **Sum**                               | `$gen_ai.usage.cache_read.input_tokens` or `$gen_ai.usage.cache_creation.input_tokens` | `$gen_ai.provider.name` |
| Reasoning-token usage    | **Sum**                               | `$gen_ai.usage.reasoning.output_tokens`                                                | `$gen_ai.provider.name` |

Bronto time-series views also provide **Count**, **Average**, **Min**, **Max**, **Sum**, and **Median**; median is the 50th percentile. Dashboard percentile views provide **P75**, **P90**, **P95**, and **P99**. Use a median to compare typical token consumption and the higher percentiles to find providers or models producing unusually large responses.

For streaming latency on spans, apply the same median and percentile views to `$gen_ai.response.time_to_first_chunk`, grouped by `$gen_ai.provider.name`. If your instrumentation exports metrics, chart `gen_ai.client.operation.time_to_first_chunk` for client-observed streaming latency, `gen_ai.client.operation.duration` for total completion time, or `gen_ai.server.time_to_first_token` for model-server TTFT.

`gen_ai.response.finish_reasons` remains an OpenTelemetry array attribute. Bronto indexes its first element as `$gen_ai.response.finish_reasons.0`; `.0` is not part of the semantic-convention name and is not normally added by the Collector.

See [Visualizations](/Search-and-Visualize/Log-Visualization) for building these views into a dashboard.

## References

* [Connect OpenTelemetry to Bronto](/agent-setup/open-telemetry)
* [OpenTelemetry GenAI semantic conventions](https://github.com/open-telemetry/semantic-conventions-genai/blob/main/docs/gen-ai/gen-ai-spans.md)
* [OpenTelemetry GenAI metric conventions](https://github.com/open-telemetry/semantic-conventions-genai/blob/main/docs/gen-ai/gen-ai-metrics.md)
* [Log Search](/Search-and-Visualize/Log-Search)
* [Custom Parser](/core-features/custom-parser)