CallShift runs AI phone agents on native speech-to-speech models (Gemini Live, OpenAI's GPT-Live and Realtime, Grok Voice) and on STT → LLM → TTS pipelines, and picks the architecture use case by use case. In production since 2024, and A/B tested against each other on a live campaign.
A speech-to-speech voice agent is an AI phone agent built on one model that hears audio and answers in audio, with no transcription and no separate voice in between. A pipeline voice agent chains three models instead: speech-to-text (STT), a large language model (LLM) and text-to-speech (TTS).
The difference is what the caller hears and what you control. Speech-to-speech keeps tone, hesitations and interruptions, and answers in a single hop. A pipeline turns every turn into text that can be inspected, constrained and replayed, at the cost of more hops and a turn-based feel.
Updated 15 October 2026
Two voice AI architectures, two opposite promises. Pipeline orchestration (STT → LLM → TTS) turns every turn of speech into text first: that is determinism. Native speech-to-speech keeps audio end to end: that is look and feel. Neither wins everywhere, and getting that arbitration right, far more than the model itself, is what decides a rollout.
One of our specialities
VAD 400 ms · STT 250 ms · LLM 350 ms · TTS 200 ms · hops 200 ms
end of speech 250 ms · first audio 500 ms
Typical figures from public benchmarks and field measurements, with the 95th percentile marked; each hop adds its own queue.
In a head-to-head A/B test run live in production, uMinds' Spanish speech-to-speech agents came out about 20% ahead of the Cartesia + GPT-4.1 mini + Deepgram pipeline they ran before. See the uMinds case
“With strong bets on models like Gemini, they’ve delivered the quality and pricing that made us move away from other providers.”
Francesco Haces MunozCofounder, uMindsThe call is made flow by flow, not once per company. What we pick, and why.
| Call flow | Architecture | Why |
|---|---|---|
| Reception, switchboard, after-hours line | Speech-to-speech | Fast answers, clean barge-in and a natural delivery: callers stay on the line. |
| Lead qualification, follow-ups, outbound campaigns | Speech-to-speech | Wherever a natural delivery converts: about 20% ahead of the pipeline in uMinds' A/B test. |
| Capturing emails, phone numbers, codes | Speech-to-speech | No transcriber in between to mangle them. |
| Long forms, conditional branching, many tools | Pipeline | Deterministic tool use and structured outputs, any LLM. |
| Regulated scripts said verbatim | Pipeline | The intermediate text is a control point: every mention checked before it is spoken. |
| Complex reasoning with a natural voice | Speech-to-speech + LLM | A full-duplex voice layer keeps the conversation while a text model thinks and calls tools. |
| Long calls, heavy prompts, predictable bill | Pipeline or flat-rate speech-to-speech | Native audio models that re-bill the whole context every turn cost more as the call grows. |
| Data kept in the EU | Pipeline, or Gemini Live and OpenAI on EU endpoints | Residency component by component; Grok Voice has no published EU option yet. |
Every model of our architecture simulator runs in production at CallShift. Scores as of 15 September 2026, Grok Voice 3.0's as of its release in October 2026.
| Model | Type | Speech to Speech Index | Typical response | Best for |
|---|---|---|---|---|
| Gemini 3.1 Flash Live | Speech-to-speech | 71.5 | ~0.8 s | Short calls with a compact prompt where the feel matters most |
| Gemini 3.8 Live | Speech-to-speech | 76.0 | ~0.8 s | Short calls with a compact prompt, the cheapest native feel on Google |
| GPT-Realtime-2.1-mini | Speech-to-speech | not scored | ~0.9 s | Simple flows where one vendor and a native feel beat fine control |
| Grok Voice 3.0 | Speech-to-speech | 86.9 | ~0.8 s | Long calls or heavy prompts at a flat, predictable price, when making the caller repeat costs the most |
| GPT-Live-1 + backend | Speech-to-speech + LLM | 81.5 | ~0.9 s | Long questionnaires and complex tasks needing reasoning, with a predictable bill |
Speech to Speech Index by Artificial Analysis. Typical response time from the caller's last word to the agent's first audio, from public benchmarks and field measurements.
Test your promptYes. uMinds runs Spanish sales calls on Gemini 3.1 Flash Live through CallShift: 50,000 calls completed in under 3 hours on their own SIP, after an A/B test in production that put speech-to-speech about 20% ahead of their Deepgram + GPT-4.1 mini + Cartesia pipeline. Read the uMinds case.
Typically in about 0.8 s from the caller's last word, 1.4 s at the 95th percentile, against about 1.4 s and 2.6 s for a pipeline. One model means one network hop; a pipeline chains voice activity detection, STT, LLM and TTS, and every hop adds its own queue.
It calls tools (your CRM, calendar, APIs) mid-call. It is not deterministic, though: it behaves like a fixed temperature of 1 and its structured output is limited. For scripts said verbatim or long forms we use a pipeline, or a speech-to-speech voice layer that delegates reasoning and tools to a text model.
Every call is recorded, transcribed and summarized after the fact, and every call is flagged and judged against the success criteria you define, with an alert after each call. The review is the same whatever the architecture.
Gemini 3.1 Flash Live has the best voice and feel of the field in every language we run, French, Spanish and Dutch included. In Spanish, Gemini 3.8 Live is close behind; in French and Dutch, the OpenAI and xAI voices trail behind.
It depends on the call. Gemini Live re-bills the prompt, the tools and all audio so far on every turn, so it suits short calls with a compact prompt; Grok Voice bills a flat $0.08 per minute; on long calls with heavy prompts a pipeline is often cheaper. Compare the three on your own prompt in the architecture simulator.
Contact us
AI agents in English, French, Dutch, Spanish and Portuguese, with localized voices, messages and spoken dates, and French tuned the deepest.
Every call recorded, transcribed, summarized and judged on your criteria, with an alert the moment it ends and a dashboard to review them.
Draw your agent's call flow in the no-code Call Builder, then drive CallShift from your stack or from Claude: a REST API, tools your agents call mid-call, an end-of-call webhook and a Claude skill.
Book a call with a voice AI engineer: we start from your real call flows and show you the agent, the architecture and the numbers that fit them.
~0% missed calls at Bonnie&Car, down from 55%. Read the case study
