OpenTelemetry

The otelMiddleware factory wires TanStack AI into your existing OpenTelemetry setup. Every chat() call produces a root span, one child span per provider model call (agent-loop turn or structured-output finalization), and one grandchild span per tool call — all with GenAI semantic-convention attributes. It also records GenAI token and duration histograms when a Meter is provided.

Structured-output calls with no tools skip the agent loop and only run the finalization request. That path still opens an iteration span (via the structuredOutput middleware phase) so backends that key off generation spans (e.g. PostHog $ai_generation) and captureContent both work. Native combined mode (supportsCombinedToolsAndSchema) does not fire that phase — the single beforeModel span covers the combined call.

Setup#

Install @opentelemetry/api — it's an optional peer dependency of @tanstack/ai:

pnpm add @opentelemetry/api

Wire up your OTel SDK however you already do (e.g. @opentelemetry/sdk-node). Then pass a Tracer (and optionally a Meter) into the middleware. The OTel middleware lives on its own subpath — importing it never affects users who don't need OTel:

import { chat } from '@tanstack/ai'
import { otelMiddleware } from '@tanstack/ai/middlewares/otel'
import { openaiText } from '@tanstack/ai-openai'
import { trace, metrics } from '@opentelemetry/api'

const otel = otelMiddleware({
  tracer: trace.getTracer('my-app'),
  meter: metrics.getMeter('my-app'),
})

const result = await chat({
  adapter: openaiText('gpt-5.5'),
  messages: [{ role: 'user', content: 'hi' }],
  middleware: [otel],
  stream: false,
})

What gets emitted#

Spans#

chat gpt-5.5              (root, kind: INTERNAL)
├── chat gpt-5.5 #0       (iteration, kind: CLIENT)
│   ├── execute_tool get_weather
│   └── execute_tool get_time
└── chat gpt-5.5 #1       (iteration, kind: CLIENT)

Iteration spans are numbered (#0, #1, ...) in the order model calls are observed, so distinct provider round-trips of the same chat are easy to pick apart in trace viewers.

Attribute reference#

Usage attributes beyond input/output tokens are emitted only when the provider reports them, so spans stay clean otherwise. Cache and reasoning breakdowns use the official GenAI semconv names; gen_ai.usage.cost and gen_ai.usage.total_tokens are de-facto extensions consumed directly by backends like PostHog — without them, backends re-derive cost from their own price tables and lose cache discounts and gateway markup. Fields with no established convention (duration-based billing, the upstream cost split) are TanStack-namespaced.

Metrics#

Two GenAI-standard histograms:

• gen_ai.client.operation.duration (seconds) — recorded once per chat() call, covering all agent-loop iterations and tool execution. On error or abort the record carries an error.type attribute (the thrown error's name, or "cancelled" for aborts).
• gen_ai.client.token.usage (tokens) — recorded once per iteration (two records: input and output), tagged with gen_ai.token.type.

Both gen_ai.response.id and gen_ai.response.model are deliberately excluded from metric attributes to keep cardinality low (per-request custom-model names and request IDs would blow up the series set).

Privacy: capturing prompts and completions#

By default, only metadata lands on spans. To record prompt and completion content, set captureContent: true. Content is captured as OTel span events following the GenAI convention:

• gen_ai.user.message, gen_ai.system.message, gen_ai.assistant.message, gen_ai.tool.message, gen_ai.choice

Pass a redact function to strip PII before anything is recorded:

import { otelMiddleware } from '@tanstack/ai/middlewares/otel'
import { trace } from '@opentelemetry/api'

const tracer = trace.getTracer('my-app')

otelMiddleware({
  tracer,
  captureContent: true,
  redact: (text) => text.replace(/\b\d{3}-\d{2}-\d{4}\b/g, '[SSN]'),
})

If redact throws, the middleware writes the literal sentinel "[redaction_failed]" into the span event and logs a warning — it never falls back to the raw content. This is the load-bearing invariant for users who ship traces to third-party backends: a broken redactor should shut off capture, not leak prompts.

Accumulated assistant text (the gen_ai.choice event) is capped at maxContentLength characters (default 100 000); longer completions are truncated with a trailing "…" marker.

Multimodal content (images, audio, video, documents) is represented as placeholder strings ([image], [audio], ...) to preserve message order without dumping binary data onto spans. Use onSpanEnd if you need richer multimodal capture.

Prompt/system/user message events fire from onConfig at the start of every iteration, which means the full conversation history (as the adapter will re-send it) is re-emitted on each iteration span. This mirrors what the provider actually sees on the wire.

Extension points#

All four extensions are optional. Each wraps user code in try/catch — a thrown callback becomes a log line, never a broken chat.

spanNameFormatter(info)#

Override default span names. info.kind is 'chat' | 'iteration' | 'tool'.

import { otelMiddleware } from '@tanstack/ai/middlewares/otel'
import { trace } from '@opentelemetry/api'

const tracer = trace.getTracer('my-app')

otelMiddleware({
  tracer,
  spanNameFormatter: (info) =>
    info.kind === 'tool' ? `tool:${info.toolName}` : `chat:${info.ctx.model}`,
})

attributeEnricher(info)#

Add custom attributes to every span. Fires once per span.

import { otelMiddleware } from '@tanstack/ai/middlewares/otel'
import { trace } from '@opentelemetry/api'
import { getCurrentTenant } from './context'

const tracer = trace.getTracer('my-app')

otelMiddleware({
  tracer,
  attributeEnricher: () => ({
    'tenant.id': getCurrentTenant(),
  }),
})

onBeforeSpanStart(info, options)#

Mutate SpanOptions immediately before tracer.startSpan(...). Useful for adding links, custom start times, or extra default attributes.

onSpanEnd(info, span)#

Fires just before every span.end(). Common uses: record custom events, emit per-tool metrics via your own Meter.

import { otelMiddleware } from '@tanstack/ai/middlewares/otel'
import { trace, metrics } from '@opentelemetry/api'

const tracer = trace.getTracer('my-app')
const meter = metrics.getMeter('my-app')

const toolDuration = meter.createHistogram('tool.duration')
otelMiddleware({
  tracer,
  onSpanEnd: (info, span) => {
    if (info.kind === 'tool') {
      // span is still recording; read timestamps from your own store if needed
      toolDuration.record(1, { 'tool.name': info.toolName })
    }
  },
})

Beyond chat: media activities#

otelMiddleware is not chat-only. The media activities — generateImage, generateVideo, generateAudio, generateSpeech, and generateTranscription — accept the same otelMiddleware value on their middleware option. Each is a single request → response (or submit → poll for video), so the middleware emits one span per call instead of the chat span tree:

import { generateImage } from '@tanstack/ai'
import { otelMiddleware } from '@tanstack/ai/middlewares/otel'
import { openaiImage } from '@tanstack/ai-openai'
import { trace, metrics } from '@opentelemetry/api'

const otel = otelMiddleware({
  tracer: trace.getTracer('my-app'),
  meter: metrics.getMeter('my-app'),
})

const result = await generateImage({
  adapter: openaiImage('gpt-image-2'),
  prompt: 'A serene mountain landscape at sunset',
  middleware: [otel],
})

The same otel value can be passed to chat() and to any media activity — its shared lifecycle hooks (onStart / onUsage / onFinish / onAbort / onError) are authored against the activity-agnostic GenerationMiddlewareContext, so the one instance works everywhere.

Each media call produces one CLIENT span tagged with the activity's gen_ai.operation.name:

The span carries gen_ai.system and gen_ai.request.model at start and, on finish, the same gen_ai.usage.* / tanstack.ai.usage.* attributes documented above — including tanstack.ai.usage.units_billed for unit-billed media. When a Meter is supplied it records the gen_ai.client.operation.duration histogram, tagged per activity. For streaming video the span covers the full create → poll → complete lifecycle. Non-streaming video is two calls, so the submit itself emits no span — the run opens once the provider accepts the job, and the getVideoJobStatus() poll that observes a terminal state ends it. If a streaming video consumer abandons the stream before completion, the span is ended via onAbort (status ERROR, tanstack.ai.completion.reason = cancelled) rather than leaked.

otelMiddleware applies the same spanNameFormatter, attributeEnricher, onBeforeSpanStart, and onSpanEnd extension points to media spans — the span info is discriminated by kind, where media spans report kind: 'generation'. For a custom backend, implement the base GenerationMiddleware contract directly; its hooks (onStart / onUsage / onFinish / onAbort / onError) receive the GenerationMiddlewareContext and fire for every activity, chat included. The GenerationMiddleware types are exported from the package root, while the otelMiddleware value lives on the @tanstack/ai/middlewares/otel subpath so importing @tanstack/ai never requires the optional @opentelemetry/api peer.

Related#

• Middleware — the lifecycle this middleware hooks into
• Debug Logging — quick console-output diagnostics, complementary to OTel