Tracing

zoxaAI integrates with Langfuse for tracing LLM calls, tool calls, and pipeline execution. Traces give you deep visibility into what happens during each call -- every LLM request, every tool invocation, and the latency of each pipeline stage.

Why Tracing Matters

Voice AI pipelines involve multiple components running in real time: speech-to-text, LLM inference, tool execution, and text-to-speech. When something goes wrong -- slow responses, incorrect answers, failed tool calls -- you need visibility into exactly what happened and where the bottleneck occurred.

Tracing provides:

• LLM call inspection -- see the exact prompts, completions, token usage, and latency for every LLM call.
• Tool debugging -- view tool invocation inputs, outputs, and duration.
• Latency analysis -- identify which pipeline stage is causing delays.
• Cost tracking -- understand per-call LLM and provider costs.
• Quality monitoring -- review conversation patterns across many calls.

How It Works

When tracing is enabled, zoxaAI exports OpenTelemetry spans to Langfuse for every pipeline run. Each span captures a discrete operation within the call pipeline. Spans are nested into a trace tree that represents the complete execution timeline for a single call.

Trace: Pipeline Run (call_a1b2c3d4)
  |-- Span: LLM Generation (GPT-4.1, 350ms)
  |-- Span: Tool Call (check_appointment, 120ms)
  |-- Span: LLM Generation (GPT-4.1, 280ms)
  |-- Span: TTS Synthesis (ElevenLabs Flash v2.5, 45ms)
  |-- Span: STT Transcription (Soniox stt-rt-v5, 30ms)

Setting Up Tracing

What Gets Traced

Each pipeline run generates spans for every major component:

Every span is tagged with the organization ID (zoxa.org_id) for filtering in Langfuse.

Viewing Traces

After tracing is configured, each call's detail page in zoxaAI includes a Trace URL that links directly to the trace in your Langfuse dashboard.

In Langfuse

Once in Langfuse, you can:

Multi-Organization Routing

zoxaAI supports per-organization Langfuse credentials. Each organization's spans are routed to their own Langfuse project independently.

There is no cross-organization data leakage -- each organization's traces are completely isolated.

Platform-Level Configuration

Platform administrators can configure a default Langfuse exporter via environment variables. These serve as the fallback when an organization does not have its own credentials configured.

Use Cases

Debugging Slow Responses

If users report that the agent takes too long to respond:

1. Open the call in Call History and click the Trace URL.
2. Look at the span durations for LLM, TTS, and STT.
3. Identify which component has the highest latency.
4. If the LLM is slow, consider switching to a faster model.
5. If TTS is slow, try a different TTS provider or voice.

Investigating Wrong Answers

If the agent gives incorrect answers:

1. Open the trace for the problematic call.
2. Inspect the LLM Generation span to see the full prompt and completion.
3. Check if the system prompt was correct and if the right context was provided.
4. Review tool call spans to see if tools returned the expected data.

Cost Optimization

1. In Langfuse, filter traces by date range and aggregate token usage.
2. Identify which calls consume the most tokens.
3. Consider optimizing system prompts to reduce token count.
4. Evaluate whether a smaller/cheaper model can handle your use case.

Troubleshooting