Zero-code instrumentation with eBPF

OpenTelemetry eBPF Instrumentation (OBI) produces distributed traces and RED metrics for services that contain no instrumentation at all — no SDK dependency, no initialisation code, and no redeploy. It attaches eBPF programs to already-running processes and reads HTTP and gRPC activity as it crosses the kernel boundary. This makes it the fastest way to get coverage across an estate, and the only practical option for services whose build you do not control: vendored binaries, legacy applications, and third-party components.

OBI is the OpenTelemetry project donated from Grafana Beyla. If you have used Beyla, the configuration keys are the same with an OTEL_EBPF_ prefix instead of BEYLA_.

When to use eBPF instead of an SDK

eBPF observes the process boundary, so it sees requests between services rather than logic inside them. The two approaches complement each other and land in Bronto through the same OTLP pipeline, with matching trace IDs.

	eBPF (OBI)	OpenTelemetry SDK
Code changes	None	Dependency plus initialisation
Redeploy needed	No	Yes
Coverage	HTTP and gRPC at the process boundary	Anything you instrument
Custom attributes	No	Yes
Business-logic spans	No	Yes
Logs	No — ship separately	Yes, via the log bridge

Use eBPF for immediate breadth, for services you cannot rebuild, and to find out where latency and errors actually are. Add an SDK in the services that then warrant domain detail. See Choose your language for SDK setup.

Requirements

Requirement	Detail
Kernel	Linux 5.8+; 5.17+ for distributed traces across services
BTF	`/sys/kernel/btf/vmlinux` must be present
Architecture	amd64 or arm64
Privileges	Root, or the documented capability set, plus host PID namespace access

Check a host before you start:

uname -r && ls -l /sys/kernel/btf/vmlinux

This also works on Docker Desktop for macOS, which runs a LinuxKit VM with BTF enabled — useful for evaluating OBI before deploying it to a cluster.

Prerequisites

A Bronto account and API key (how to create one)
An OpenTelemetry Collector — recommended, and required if you want delta metrics
A host or cluster meeting the requirements above

Run OBI with Docker Compose

OBI runs as one container alongside your services. It needs the host PID namespace to see other containers’ processes, and elevated privileges to load eBPF programs.

docker-compose.yml

services:
  obi:
    image: otel/ebpf-instrument:main
    pid: "host"
    privileged: true
    environment:
      OTEL_EBPF_CONFIG_PATH: /etc/obi/obi.yml
    volumes:
      - ./obi.yml:/etc/obi/obi.yml:ro
    depends_on: [otel-collector]

OBI discovers what to instrument by the ports your services listen on — the applications themselves are not modified or restarted:

obi.yml

discovery:
  instrument:
    # The ports your services listen on.
    - open_ports: 8080,3000,5000
      containers_only: true

  exclude_instrument:
    # Publishing a port makes the Docker daemon listen on it too, so a
    # port selector matches dockerd unless you exclude it explicitly.
    - exe_path: "*dockerd*"
    - exe_path: "*docker-proxy*"
    - exe_path: "*containerd*"
    - exe_path: "*otelcol*"

ebpf:
  # Inject W3C trace context into outgoing requests so spans from different
  # services join into one distributed trace. Requires kernel 5.17+.
  context_propagation: all

otel_traces_export:
  endpoint: http://otel-collector:4318

otel_metrics_export:
  endpoint: http://otel-collector:4318
  interval: 15s
  features:
    - application
    - application_span

Run OBI on Kubernetes

Deploy OBI as a DaemonSet so every node instruments its own pods. The container needs hostPID: true and the eBPF capabilities:

obi-daemonset.yaml

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: obi
spec:
  selector:
    matchLabels: { app: obi }
  template:
    metadata:
      labels: { app: obi }
    spec:
      hostPID: true
      containers:
        - name: obi
          image: otel/ebpf-instrument:main
          securityContext:
            privileged: true
          env:
            - name: OTEL_EBPF_CONFIG_PATH
              value: /etc/obi/obi.yml
            - name: OTEL_EBPF_BPF_CONTEXT_PROPAGATION
              value: all
          volumeMounts:
            - { name: config, mountPath: /etc/obi }
      volumes:
        - name: config
          configMap: { name: obi-config }

On Kubernetes you can select workloads by namespace, labels, or annotations instead of ports:

obi.yml

discovery:
  instrument:
    - k8s_namespace: production
      k8s_pod_labels:
        instrument: obi

See Kubernetes logs and metrics with OpenTelemetry for collecting pod logs alongside this.

Forward to Bronto

Point the Collector’s OTLP receiver at Bronto’s trace and metric endpoints. Traces and metrics use separate endpoints and the same API key header:

otel-config.yaml

receivers:
  otlp:
    protocols:
      grpc: { endpoint: 0.0.0.0:4317 }
      http: { endpoint: 0.0.0.0:4318 }

processors:
  batch:
  # OBI exports cumulative metrics; Bronto strongly prefers delta.
  cumulativetodelta:
    initial_value: drop

exporters:
  otlphttp/brontotraces:
    traces_endpoint: "https://ingestion.<REGION>.bronto.io/v1/traces"
    compression: gzip
    headers:
      x-bronto-api-key: <YOUR_API_KEY>
  otlphttp/brontometrics:
    metrics_endpoint: "https://ingestion.<REGION>.bronto.io/v1/metrics"
    compression: gzip
    headers:
      x-bronto-api-key: <YOUR_API_KEY>

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlphttp/brontotraces]
    metrics:
      receivers: [otlp]
      processors: [cumulativetodelta, batch]
      exporters: [otlphttp/brontometrics]

Replace <REGION> with eu or us to match your account. A mismatched region returns 401, which looks identical to an invalid key.

Do not set histogram_aggregation: base2_exponential_bucket_histogram. Bronto does not currently ingest exponential histograms. OBI’s default, explicit_bucket_histogram, is the correct setting.

Name your services

By default OBI names a service after its executable. That is fine for a compiled binary, but interpreted runtimes all end up named after the interpreter — node, python3.13, java. OBI reads OTEL_SERVICE_NAME and OTEL_RESOURCE_ATTRIBUTES from each target process’s own environment, so you can name services without adding an SDK or touching application code:

docker-compose.yml

services:
  checkout-api:
    image: your-app:latest
    environment:
      OTEL_SERVICE_NAME: checkout-api
      OTEL_RESOURCE_ATTRIBUTES: service.namespace=production

Nothing inside the container reads these variables — OBI does, from the outside.

Traces are stored in Bronto’s .traces collection, with one dataset per service.name. service.namespace is carried on every span and is the attribute to filter on to isolate an environment, but it does not determine where traces are stored.

What you will see in Bronto

OBI emits standard OpenTelemetry semantic conventions, so no Bronto-specific configuration is required. Traces. Each request produces a server span, client spans for outgoing calls, and — for Go services — an in queue and processing breakdown that separates time spent waiting to be handled from time spent being handled:

checkout-api    SERVER    GET /checkout    50.65ms
checkout-api    INTERNAL  in queue          0.20ms
checkout-api    INTERNAL  processing       50.45ms
checkout-api    CLIENT    GET /orders      49.93ms
  orders-api    SERVER    GET /orders      48.28ms
  orders-api    CLIENT    GET /inventory   46.54ms
    inventory   SERVER    GET /inventory   46.35ms

Metrics. RED metrics per route and status code:

http.server.request.duration
http.server.request.body.size
http.server.response.body.size
http.client.request.duration
traces_spanmetrics_calls_total
traces_spanmetrics_latency

Searchable span fields. span.trace_id, span.parent_span_id, span.kind, span.duration_nano, http.route, http.request.method, http.response.status_code, service.name, service.namespace, and url.path. Spans produced by OBI carry telemetry.distro.name = opentelemetry-ebpf-instrumentation, which is a convenient way to separate eBPF-derived data from SDK-derived data in queries.

Verify

Confirm OBI attached to your processes

OBI logs one line per instrumented process, including the runtime it detected:

instrumenting process cmd=/usr/local/bin/checkout-api type=go
instrumenting process cmd=/usr/local/bin/node          type=nodejs
instrumenting process cmd=/usr/local/bin/python3.13    type=python

If nothing appears, your selector matched no processes — check open_ports against the ports your services actually listen on.

Print spans without leaving the terminal

Set trace_printer: text in obi.yml to have OBI print every captured span to stdout as well as exporting it. Send a request through your service and confirm spans appear, then set it back to disabled.

Check delivery to Bronto

Look for export errors in the Collector’s logs:

docker compose logs otel-collector | grep -i "exporting failed"

A 401 means the API key or the region is wrong.

Find the data in Bronto

Open Explore Traces and filter on your service.name, or query service.namespace to see the whole environment. Metrics appear in the Metric Explorer under http.server.request.duration.

Troubleshooting

Spans appear but never join into a trace. Context propagation is off by default. Set context_propagation: all and confirm the kernel is 5.17+.
A service you did not expect is being instrumented. Publishing a container port makes the Docker daemon listen on it too. Add dockerd, docker-proxy, and containerd to exclude_instrument.
Go services produce fewer details than expected. OBI attaches uprobes to Go runtime symbols. Building with -ldflags="-s -w" strips the symbol table and silently degrades OBI to generic syscall tracing — leave Go binaries unstripped.
creating OTEL namespace in bpffs failed. /sys/fs/bpf is not mounted. Core tracing still works; features that rely on pinned maps, such as the log enricher, are disabled. Mount bpffs to restore them.
FIONREAD compensation is ineffective. Applications that size reads via FIONREAD — nginx, Java, and .NET among them — may stall or truncate transfers while context propagation is enabled. Validate against those runtimes, or set context_propagation: disabled for them.
Metrics rejected or missing. Confirm you are not exporting exponential histograms, and that cumulativetodelta is in the metrics pipeline.

Security considerations

Running with privileged: true is the documented starting point and the simplest way to prove the setup works, but it grants more than OBI needs. For production, use the least-privilege capability set — CAP_BPF, CAP_PERFMON, CAP_NET_RAW, and CAP_DAC_READ_SEARCH among others, depending on the features you enable. Set OTEL_EBPF_ENFORCE_SYS_CAPS=true so OBI fails loudly when a required capability is missing instead of degrading quietly. Host PID namespace access is not optional: OBI must see other processes in order to attach to them.

Go library-level context propagation relies on bpf_probe_write_user, which is blocked by Secure Boot and kernel lockdown mode. Network-level propagation still works in those environments.

Next steps

Explore Traces — investigate services, latency, and errors
Send Metrics to Bronto — metric support and limitations
Choose your language — add SDK instrumentation where you need business context
Agent Setup — ship logs, which OBI does not produce
OBI documentation — full configuration reference

Overview

Ingest with agents

Ingest from AWS

Ingest from Azure

Ingest from CDN and edge

Ingest from apps and platforms

Ingest from infrastructure and databases

Ingest OpenTelemetry data

Zero-code instrumentation with eBPF

When to use eBPF instead of an SDK

Requirements

Prerequisites

Run OBI with Docker Compose

Run OBI on Kubernetes

Forward to Bronto

Name your services

What you will see in Bronto

Verify

Troubleshooting

Security considerations

Next steps

​When to use eBPF instead of an SDK

​Requirements

​Prerequisites

​Run OBI with Docker Compose

​Run OBI on Kubernetes

​Forward to Bronto

​Name your services

​What you will see in Bronto

​Verify

​Troubleshooting

​Security considerations

​Next steps

When to use eBPF instead of an SDK

Requirements

Prerequisites

Run OBI with Docker Compose

Run OBI on Kubernetes

Forward to Bronto

Name your services

What you will see in Bronto

Verify

Troubleshooting

Security considerations

Next steps