Silero VAD and Smart Turn solve different parts of turn-taking

Silero VAD tells you someone stopped making sound. Pipecat's Smart Turn tells you whether they stopped talking. A Python voice agent needs both answers.

Duplex Compute · October 13, 2026 · Versions checked October 13, 2026

A voice agent that answers after a fixed silence will answer "I can't seem to, um..." before the user finds the words. That example comes from the Smart Turn README, which suggests trying it against "I can't seem to, um, find the return label." The two phrases have the same pause. Only one of them is finished.

Silero VAD and Smart Turn split this problem. Silero runs on every small chunk of audio and marks speech and silence. Smart Turn runs only when Silero reports silence, reads the whole current turn, and returns the probability that the speaker is done.

The Silero figures come from the Silero README and the chunk check in its source, which rejects any other chunk size. The Smart Turn figures come from its README. Both speed figures are the authors' numbers on their hardware. Neither is the latency your user hears, which also includes transport, the language model and speech synthesis.

Pipecat publishes a benchmark report for the v3.2 CPU model: 92.63% accuracy on 31,527 samples across 23 languages. It is the authors' own test set. Your callers, microphones and languages will move that number.

How the two stages fit

mic audio (any rate, any channels)
   |
   v
resample to 16 kHz mono float32
   |
   v
Silero VAD, every 512 samples -----> speech start: keep recording the turn
   |
   | speech end (after ~200 ms of silence)
   v
Smart Turn on the whole turn so far
   |
   +-- p > 0.5 --> end of turn: reply, reset both stages
   +-- p <= 0.5 -> keep listening; if speech resumes, rerun on the longer turn
                    if silence passes the fallback (3 s), end the turn anyway

The short VAD silence matters. Pipecat's Smart Turn docs recommend a VAD stop time of 0.2 s, because Smart Turn now makes the real decision. They default the fallback silence to 3 s, so a user who stops mid-sentence and walks away still gets an answer.

A Python sketch

This is a sketch built from the two repositories' code. We have not run it. Package versions are the latest on PyPI on October 13, 2026 for silero-vad (6.2.3), and the transformers and onnxruntime versions pinned in Smart Turn's requirements.txt (which pins the GPU build; this uses the CPU build of the same version).

# pip install silero-vad==6.2.3 onnxruntime==1.23.2 transformers==4.48.2 scipy numpy
# Download smart-turn-v3.2-cpu.onnx from huggingface.co/pipecat-ai/smart-turn-v3
from collections import deque
from math import gcd

import numpy as np
import onnxruntime as ort
import torch
from scipy.signal import resample_poly
from silero_vad import VADIterator, load_silero_vad
from transformers import WhisperFeatureExtractor

SR = 16000
CHUNK = 512            # Silero accepts exactly 512 samples per call at 16 kHz
MAX_TURN_S = 8         # Smart Turn's input window


def to_16k_mono(pcm: np.ndarray, sr: int) -> np.ndarray:
    """int16 or float PCM, shape (n,) or (n, channels) -> float32 mono at 16 kHz."""
    x = pcm.astype(np.float32)
    if pcm.dtype == np.int16:
        x /= 32768.0
    if x.ndim == 2:
        x = x.mean(axis=1)
    if sr != SR:
        g = gcd(sr, SR)
        x = resample_poly(x, SR // g, sr // g).astype(np.float32)
    return x


class SmartTurn:
    def __init__(self, path="smart-turn-v3.2-cpu.onnx"):
        self.session = ort.InferenceSession(path)
        self.features = WhisperFeatureExtractor(chunk_length=MAX_TURN_S)

    def probability(self, turn: np.ndarray) -> float:
        n = MAX_TURN_S * SR
        x = turn[-n:]                         # keep the end of long turns
        x = np.pad(x, (n - len(x), 0))        # zero-pad at the start
        f = self.features(x, sampling_rate=SR, return_tensors="np",
                          padding="max_length", max_length=n,
                          truncation=True, do_normalize=True)
        inputs = {"input_features": f.input_features.astype(np.float32)}
        return float(self.session.run(None, inputs)[0][0].item())


class TurnDetector:
    def __init__(self, threshold=0.5, fallback_silence_s=3.0):
        self.vad = VADIterator(load_silero_vad(), sampling_rate=SR,
                               threshold=0.5, min_silence_duration_ms=200)
        self.smart_turn = SmartTurn()
        self.threshold = threshold
        self.fallback_samples = int(fallback_silence_s * SR)
        self.reset()

    def reset(self):
        """Call after every completed turn and whenever the agent is interrupted."""
        self.vad.reset_states()               # Silero's recurrent state + iterator flags
        self.pending = np.zeros(0, dtype=np.float32)
        self.preroll = deque(maxlen=7)        # ~220 ms kept before speech starts
        self.turn = []
        self.speaking = False
        self.silence = 0

    def feed(self, pcm: np.ndarray, sr: int) -> bool:
        """Feed raw audio; return True when the user's turn has ended."""
        self.pending = np.concatenate([self.pending, to_16k_mono(pcm, sr)])
        while len(self.pending) >= CHUNK:
            chunk, self.pending = self.pending[:CHUNK], self.pending[CHUNK:]
            event = self.vad(torch.from_numpy(chunk)) or {}

            if "start" in event:
                if not self.turn:
                    self.turn.extend(self.preroll)
                self.speaking, self.silence = True, 0
            if self.turn or self.speaking:
                self.turn.append(chunk)
            else:
                self.preroll.append(chunk)

            if "end" in event:
                self.speaking = False
                p = self.smart_turn.probability(np.concatenate(self.turn))
                if p > self.threshold:
                    return True
            elif self.turn and not self.speaking:
                self.silence += CHUNK
                if self.silence >= self.fallback_samples:
                    return True               # fallback: stop waiting
        return False

The caller loop is short: feed each audio frame from your transport, and when feed returns True, hand np.concatenate(detector.turn) to your speech-to-text or speech model, then call reset().

Resampling to 16 kHz mono

Your transport may hand you 48 kHz stereo from a browser or 8 kHz audio from a phone line. Silero accepts 8 kHz or 16 kHz. Smart Turn takes only 16 kHz. Resampling once at the front keeps one sample rate for both models. scipy.signal.resample_poly applies an anti-aliasing filter, so 48 kHz to 16 kHz is a 1:3 polyphase step. Do not drop every third sample by slicing. Average stereo channels before the models see them. Convert int16 to float in the range −1 to 1; both models expect float audio.

Reset state per turn

Silero is a recurrent model. Its VADIterator.reset_states() clears the model's hidden state along with the iterator's speech flags. Smart Turn's own recording demo also resets Silero's state every five seconds. Reset at every turn boundary and when the user barges in, or the next turn starts with stale state and a half-built buffer.

Smart Turn has no state between calls. It needs the right audio. The README asks for the whole current turn, trimmed from the start to about 8 seconds and zero-padded at the beginning when shorter. If the user starts speaking again before Smart Turn finishes, rerun it on the entire turn, not on the new segment. Previous turns do not need to be included.

Testing it on pauses and corrections

Build a small fixture set before you tune thresholds. Record each clip a few times, with your real microphone path:

• "I can't seem to, um..." with a 1–2 s pause, then "find the return label."
• "I want to fly to Boston." pause "On Friday."
• "Book the 9 a.m. No, the 10 a.m." with no pause at the correction.
• A card or phone number read in groups with gaps.
• A cough, a door, and a TV in the background.

For each clip, log when Silero reported the end of speech, each Smart Turn probability, and when the detector returned. Then check whether the agent's reply used the full request. A detector that answers 300 ms sooner and loses "On Friday" made the task fail. Our semantic VAD note has an illustrative timing table for the hesitant case, and the evaluation note covers scoring.

When a provider owns endpointing

With a speech-to-speech API, the provider's detector usually decides turns, and your local pipeline has nothing to gate. Pipecat's interruptions page says that for services like OpenAI Realtime and Gemini Live, the provider detects barge-in and Pipecat follows its events. LiveKit lets you turn off OpenAI Realtime's server detection and drive turns with Silero plus LiveKit's own turn detector; it notes that Gemini Live does not support that mode. The OpenAI equivalent of this two-stage detector is semantic_vad, covered in the semantic VAD note.

In Pipecat itself you rarely write the code above. Since v0.0.102, a turn analyzer stop strategy using LocalSmartTurnAnalyzerV3 is the default way Pipecat ends user turns, and the weights ship with the package. The sketch is for custom pipelines and for understanding what that default does.

Both stages exist because a turn-based agent must pick one moment to stop listening and start answering. A full-duplex model keeps listening while it speaks, so it still hears a late "on Friday" after it has started talking. Duplex Compute is a full-duplex voice API with tool calling, at $0.01 per connected minute. If your detector tuning keeps trading cut-offs for slow replies, our full-duplex explainer compares the two designs.

Related notes

• Semantic VAD vs. server VAD: stop cutting off unfinished sentences
• Why your Realtime agent keeps talking after an interruption
• Pipecat vs. LiveKit
• Full-duplex voice AI vs. turn-based