OpenTelemetry eBPF Instrumentation Reaches v1.0.0

OpenTelemetry eBPF Instrumentation (OBI) hit v1.0.0 on October 30, 2026. It watches Linux processes from inside the kernel and turns what it sees into OpenTelemetry traces and metrics, so a service you can't edit still shows up in your tracing backend. It started as Grafana Beyla, which Grafana donated to the OpenTelemetry project.

If you deploy AI into code you don't own, that is a handy trick, but it needs kernel-level privileges, so this post covers the news and the security review.

What shipped in OpenTelemetry eBPF Instrumentation 1.0

The v1.0.0 release notes call it OBI's first stable release. It promotes v1.0.0-rc.2 and publishes the 1.0.0 telemetry schema.

What 1.0 promises, per the notes and the announcement post:

• Stable surfaces: Config v2, documented CLI flags, environment variables, release binaries, container behavior, the root Go module's public API, and telemetry classified stable. Incompatible changes need a new major version.
• Config v1 is frozen. It works but gets no fixes or new features. Config v2 (since v0.11.0) adds obi config validate and obi config migrate.
• Not everything is stable. Telemetry whose conventions are still developing may change, and runtime metrics for Go, Java, .NET, Node.js and Python are experimental.
• Supply chain artifacts: release archives and images, signature bundles, CycloneDX SBOMs and a release attestation. The announcement credits this tooling to rc.2, so treat it as practice, not a promise.

OpenTelemetry LLM tracing with zero code changes

Per the OBI docs, GenAI coverage includes OpenAI, OpenAI-compatible gateways, Ollama, Anthropic Claude, Google AI Studio (Gemini), AWS Bedrock, embedding and rerank APIs, vector retrieval and MCP over JSON-RPC. The 1.0 announcement adds tool-call and retrieval instrumentation.

Capturing request and response content needs explicit payload-capture configuration and is off by default. I couldn't confirm the exact keys, so take them from the Config v2 reference and run obi config validate.

OBI sees what crosses process boundaries. For custom spans or business attributes, the docs still point you to language agents or manual instrumentation.

Check the privileges before you run an eBPF agent

OBI needs Linux 5.8+ (RHEL-family 4.18+ with backports), BTF, and amd64 or arm64, plus root or the Linux capabilities for the features you enable. Review it like this:

1. Skip privileged: true. The docs show an unprivileged DaemonSet that drops all capabilities, then adds BPF, SYS_PTRACE, NET_RAW, CHECKPOINT_RESTORE, DAC_READ_SEARCH and PERFMON.
2. Treat SYS_ADMIN as opt-in. It is only needed for bpf_probe_write_user() features such as Go library-level context propagation, which also wants NET_ADMIN.
3. Fail loudly. With OTEL_EBPF_ENFORCE_SYS_CAPS=1, OBI stops at startup if a capability is missing. Otherwise it logs a warning and carries on.
4. Check the node. On kernels below 5.11, add SYS_RESOURCE. On 6.6.128+, 6.12.75+, 6.18.14+ and 6.19+, mount the host's /sys/kernel/tracing or context propagation is disabled.
5. Verify what you pull. Check the signature bundles and SBOMs and pin the image digest. The release page says the tag-signing key has expired, so don't trust tag signatures alone.

On a standalone host, grant the same set to the binary (add cap_net_admin,cap_sys_admin only for context propagation):

sudo setcap cap_bpf,cap_checkpoint_restore,cap_dac_read_search,cap_perfmon,cap_net_raw,cap_sys_ptrace+ep ./bin/obi
OTEL_EBPF_ENFORCE_SYS_CAPS=1 OTEL_EBPF_CONFIG_PATH=<path-to-config>.yml ./bin/obi

Roll it out as a canary

The release notes say to follow the Config v1 to v2 migration guide, review environment-variable configuration, and compare telemetry in a canary first. A DaemonSet pinned to one labeled node does that:

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: obi-canary
  namespace: observability
spec:
  selector:
    matchLabels:
      app: obi-canary
  template:
    metadata:
      labels:
        app: obi-canary
    spec:
      hostPID: true # required to see processes on the host
      serviceAccountName: obi
      nodeSelector:
        obi-canary: "true"
      containers:
        - name: obi
          image: otel/ebpf-instrument:<pinned-version> # the docs example still shows a v0.15.2 tag; pin one you have verified
          securityContext:
            runAsUser: 0
            readOnlyRootFilesystem: true
            capabilities:
              drop:
                - ALL
              add:
                - BPF
                - SYS_PTRACE
                - NET_RAW
                - CHECKPOINT_RESTORE
                - DAC_READ_SEARCH
                - PERFMON
          env:
            - name: OTEL_EBPF_ENFORCE_SYS_CAPS
              value: "1"
            - name: OTEL_EBPF_AUTO_TARGET_EXE
              value: "*/<your-service-binary>"
            - name: OTEL_EXPORTER_OTLP_ENDPOINT
              value: "http://otelcol:4318"
            - name: OTEL_EBPF_KUBE_METADATA_ENABLE
              value: "autodetect"
            # keep OTLP credentials in a Secret, not a ConfigMap
            - name: OTEL_EXPORTER_OTLP_HEADERS
              valueFrom:
                secretKeyRef:
                  name: otlp-credentials
                  key: headers
          volumeMounts:
            - name: tracefs # needed only on the newer kernels listed above
              mountPath: /sys/kernel/tracing
      volumes:
        - name: tracefs
          hostPath:
            path: /sys/kernel/tracing
            type: Directory

This follows the unprivileged example in the Kubernetes setup docs. Create the obi ServiceAccount and RBAC from that page too; metadata decoration depends on them.

kubectl label node <node-name> obi-canary=true
kubectl apply -f obi-canary.yaml
kubectl -n observability logs ds/obi-canary --tail=50

The logs show whether it started with the capabilities it needs.

What to do on Monday morning

1. Pick a low-risk node pool and run the canary against one service with payload capture off, since prompts can hold customer data.
2. Compare against your current telemetry. Upgrading from before v0.15.2? The notes flag changes to span names, rpc.method values, default metric attributes and Prometheus labels that can break dashboards.
3. Hand the capability list to your security reviewer before you widen the rollout.

Gotchas

• Known issues on the release page: application metrics can go missing after a Pod starts or restarts (restart OBI to recover), child spans can start before their parents, and port-based discovery can pick a helper process.

For anyone running AI in production, 1.0 means tracing model, retrieval and MCP calls on code you don't own, if you can defend the privileges it takes.