Full-duplex voice AI: listening while speaking, and what turn-based agents do instead

A full-duplex voice model hears the user while it talks. A turn-based agent waits for the end of a turn, and stops talking when someone barges in. Both can be interrupted. They differ in what they do with speech that overlaps their own.

Duplex Compute · October 13, 2026 · Rates and versions checked October 13, 2026

OpenAI describes GPT-Live as able to listen while speaking, which it calls full duplex. Kyutai's Moshi makes the idea concrete in its architecture: it models two audio streams, one for its own speech and one for the user's, and predicts text for its own speech alongside them. Neither stream waits for the other. The model keeps processing the user's audio while it produces its own.

A turn-based agent alternates. It listens, decides the user has finished, generates a reply and speaks it. Pipelines built on speech-to-text, an LLM and text-to-speech work this way. So does gpt-realtime-2.1, whose sessions run as a series of turns, each producing one response.

The term "full duplex" also has an older telecom meaning: a line that carries audio both ways at once. Every WebRTC call is full duplex in that sense. A bidirectional WebSocket or WebRTC connection shows the transport can carry overlapping audio. It says nothing about whether the model uses it.

How turn-based agents handle timing

A turn-based agent needs to answer two questions about the user's audio. Has the user finished? And has the user started talking over me?

The first is end-of-turn detection. OpenAI's server_vad mode ends a turn after a period of silence set by silence_duration_ms. Shorter values respond sooner and cut off more people who were only pausing. semantic_vad adds a classifier that scores whether the words so far sound finished, and waits longer when they don't. LiveKit Agents uses voice activity detection plus endpointing heuristics, optionally with a model that reads the meaning of the speech.

The second is barge-in. In Realtime, with voice activity detection on, the API detects user speech, cancels the ongoing response and starts a new one. Over WebRTC and SIP the server truncates the audio the user never heard. Over WebSocket your client has to stop playback and send the truncation itself. Pipecat's interruption flow cancels in-flight LLM and TTS work, flushes unplayed audio and keeps only the words that were actually spoken in the context.

Barge-in is a stop signal. It works well when the user wants the agent to stop. It's a blunt tool when the user only meant to agree or add a detail. LiveKit documents this case: when voice activity detection fires and the transcript comes back empty, it treats that as a false interruption and can resume speaking where it left off.

Three overlaps that separate the two

Overlapping speech means different things. A system that treats all of it as "stop" handles only one of these well.

The "can" in the full-duplex column describes a capability. A model that hears overlapping speech still has to judge it correctly. Full duplex removes a structural limit. Whether a given model makes good use of it is something to test.

A timeline of the correction case shows where the two designs part.

Tools while the conversation continues

Overlap matters most when the agent is doing something: checking availability, placing an order. Here the designs vary more than the duplex label suggests.

GPT-Live sends tool work to a separate backend. Users can keep talking while backend work runs, and the backend can continue when the caller interrupts. Your application decides whether to finish or cancel it.

NVIDIA's full-duplex NemotronLabs VoiceChat has a tool channel, and its README lists the limits plainly. The user cannot interrupt the agent while a tool runs. Long tool responses can delay speech. Simultaneous tool calls are unreliable, and NVIDIA recommends at most five tools per session. A model can be full duplex in conversation and still block during a tool call.

Turn-based frameworks have their own switches. In Pipecat, a function with cancel_on_interruption=True, the default, is cancelled when the user interrupts and holds the conversation while it runs. Setting it to False makes the call asynchronous, so the conversation carries on.

Whichever design you pick, your application owns side effects. Stopping the voice does not undo a booking. Give each action an idempotency key, and decide in code what a correction does to work already running.

What a benchmark measures

Full-Duplex-Bench-v3 tests this with real human audio annotated for five kinds of disfluency, in scenarios that need chained API calls across four task domains. It scores accuracy, latency and turn-taking. In the paper's results, GPT-Realtime led on Pass@1 at 0.600. Gemini Live 3.1 had the fastest latency, 4.25 s, and the lowest turn-take rate, 78.0%. A cascaded Whisper, GPT-4o and TTS pipeline took every turn and had the highest latency, 10.12 s. The authors report self-correction handling and multi-step reasoning as the most common failures across all systems.

Those are the paper's numbers for the model versions it tested. They don't describe newer releases. Our note on reading voice benchmarks explains how to read them.

Choosing between them

A turn-based agent with well-tuned turn detection and barge-in suits short, structured exchanges: a question, an answer, a confirmation. Its failure modes are known and the controls are documented.

Full duplex helps most in long, loose conversations: tutoring, coaching, support calls where the user thinks aloud, agrees as you go and corrects details mid-sentence. Those are the moments where "stop" is the wrong reading of overlapping speech.

Test with your own audio either way. Record a backchannel, a correction and a real stop, put the user and agent audio on one timeline, and check the application state afterwards, as OpenAI's voice agent evaluation cookbook recommends.

Duplex Compute is a full-duplex voice API with tool calling, configurable behavior and long conversations, at $0.01 per connected minute. It fits the second group of applications. Your application still owns the tools and their side effects.

Related notes

• Semantic VAD vs. silence-based turn detection
• Why your Realtime agent keeps talking after an interruption
• GPT-Live vs. Realtime API
• Voice agent evaluation
• Nemotron VoiceChat tool calling