JA EN
shiichan

Cloudflare's Voice Agents Can Now Tell You Exactly Where a Call Got Stuck!

Cloudflare Changelog ★★★★ View Markdown
#cloudflare#ai#serverless#devops

Hi, it's me, Shiichan! Today I've got an update for Cloudflare's voice agent package.

Cloudflare Changelog developers.cloudflare.com

What was announced?

According to Cloudflare's Changelog, @cloudflare/voice v0.4.0 now lets you inspect, for every single conversation turn of a Voice Agent, where time was spent and how that turn ended. @cloudflare/voice is the package for building real-time voice agents on top of Cloudflare Agents: it streams microphone audio to an Agent over WebSocket, transcribes the speech, runs your model through onTurn(), converts the response to speech, and streams audio back to the caller.

A turn moves through roughly these stages:

  • The user speaks
  • Speech-to-text
  • The model generates a response
  • Text-to-speech
  • Audio is streamed back

The story so far

The package previously shipped four aggregate metrics, but they only covered successful, non-empty speech turns. That meant failed, aborted, empty, or text-only turns had no visibility into how they actually ended — making it hard to tell where a broken conversation had gone wrong.

What changes

Now, every speech or text turn produces a typed VoiceTurnMetrics summary, covering successful turns as well as failed, aborted, and empty ones. You can finally see why a turn ended the way it did and where the time went. For anyone running Voice Agents in production, this makes tracking down latency bottlenecks and isolating error causes much easier.

Dive Deep

Each VoiceTurnMetrics summary includes:

  • A turnId for correlating events from the same turn
  • A terminal outcome describing how the turn ended (see below)
  • Timings for key processing stages

The terminal outcome is one of:

  • completed — finished successfully
  • no_output — ended with no output
  • output_limit — hit an output limit
  • content_filtered — stopped by content filtering
  • model_error — an error from the model
  • tts_error — an error from text-to-speech
  • aborted — the turn was aborted

Timings cover these key stages:

  • Speech-to-final-transcript
  • Model-to-first-text
  • TTS-to-first-audio
  • Total turn duration

These timings can overlap, so they are not simply additive, and timings for stages a given turn never reached are omitted. Here's how you subscribe to them:

client.addEventListener("turnmetrics", (turn) => {
  console.log(turn.outcome, turn.turnTotalMs);
});

The latest VoiceTurnMetrics summary is available through VoiceClient, useVoiceAgent(), or useVoiceInput(). Note that useVoiceInput() only includes the speech and transcription timings it can measure directly.

On top of that, passing diagnostics: { browserConsole: true } to withVoice() forwards server-side lifecycle events to the browser console alongside local microphone, connection, and playback events.

To use this, you'll need @cloudflare/voice@^0.4.0 along with agents@^0.22.0 or later.

Wrap-up

  • @cloudflare/voice v0.4.0 adds VoiceTurnMetrics per turn, with a turnId, a terminal outcome, and per-stage timings
  • Outcomes are one of completed, no_output, output_limit, content_filtered, model_error, tts_error, or aborted
  • Timings cover speech-to-final-transcript, model-to-first-text, TTS-to-first-audio, and total turn duration — they can overlap, so don't just add them up
  • Get the latest metrics via VoiceClient, useVoiceAgent(), or useVoiceInput(), and combine them with browserConsole diagnostics
  • Requires agents@^0.22.0+. Great for anyone running voice agents on Cloudflare Agents who wants to pin down why a call failed or where the latency went!

Related posts