Self-hosted voice AI vs. managed APIs: what a GPU conversation-hour costs
An open full-duplex model on your own GPU costs the GPU's hourly price divided by the conversations it actually serves. Few teams know that second number before they try.
"Self-hosted voice AI" covers three different projects. Running Pipecat or LiveKit Agents on your own servers is one. It moves orchestration in-house while STT, the LLM and TTS still bill per call to their vendors. Running open STT and TTS models next to a hosted LLM is another. The third, and the subject here, is running a speech-to-speech model on your own GPU, so no per-minute voice bill exists at all. This note covers two open models you can run that way.
The two models
| NemotronLabs VoiceChat 11B | Moshi | |
|---|---|---|
| Publisher | NVIDIA | Kyutai |
| Size | 11B, built on Nemotron-Nano-9B-v2 | 7B Temporal Transformer plus a small Depth Transformer |
| GPU memory | 80 GB GPU required; the container needs about 66 GB | About 24 GB for the PyTorch version; MLX runs on a Mac |
| Serving paths | Offline Hugging Face checkpoint, or an NVIDIA container (CUDA, Triton, vLLM) with a WebSocket API | PyTorch, MLX, Rust (the Rust path is labelled "for production") |
| Tool calling | Separate tool-call output channel; at most 5 tools recommended | No tool interface in the README |
| Code licence | Apache 2.0 | MIT (Python, web client), Apache (Rust) |
| Weights licence | OpenMDW 1.1 | CC-BY 4.0 |
Sources: the NemotronLabs VoiceChat README, its container deployment guide for the 66 GB figure, and the Moshi README.
Kyutai reports a theoretical latency of 160 ms for Moshi and a practical latency as low as 200 ms on an L4 GPU. Those are the authors' numbers on their setup. NVIDIA's README gives no latency figure for VoiceChat.
Read the known limitations before you price anything. NVIDIA writes that VoiceChat was trained on audio contexts of no more than two minutes, and context beyond that may not be retained. The user cannot interrupt the agent while a tool runs. The model cannot reliably call several tools at once. It ships one fixed voice. The README also lists self-talk after a finished turn, word loops, and cutting the user off at mid-sentence pauses. Some products can live with those. A two-hour tutoring session or a five-tool support agent probably can't.
A hosted wrapper can expose less than the checkpoint supports. .wave serves Nemotron VoiceChat at an introductory $0.005 per minute, in public beta, with sessions capped at 120 seconds and no tools, custom instructions or text input. Self-hosting gets you the tool channel. It also gets you the work of wiring tool results back into a live session.
The break-even formula
GPU cost per conversation-hour = GPU $/hour ÷ (concurrent sessions per GPU × average slot utilization)Concurrent sessions per GPU is the number you have to measure. VRAM won't tell you. NVIDIA's container client has a --num-streams flag for load tests, but the guide publishes no sessions-per-GPU figure. You find it by running streams until latency or quality breaks. Utilization is the share of paid GPU time with a caller on the line. Traffic arrives in peaks, and you keep warm capacity for the peak.
Add what the formula leaves out: CPU and memory for the session server, bandwidth, model load time on scale-up, logging and monitoring, any LLM you call for harder reasoning, failed sessions you retry, and the engineer who gets paged.
A worked example with made-up inputs
Hypothetical. The $2 per GPU-hour price and the four sessions per GPU are assumptions chosen to show the arithmetic. They are not a quote, and not a measured Nemotron or Moshi result.
| Average utilization | Conversation-hours per GPU-hour | GPU cost per conversation-hour |
|---|---|---|
| 100% | 4.0 | $0.50 |
| 75% | 3.0 | $0.67 |
| 50% | 2.0 | $1.00 |
| 25% | 1.0 | $2.00 |
Compare that with managed session rates. GPT-Live 1 is $0.05 per minute, or $3.00 per hour, with backend model usage billed on top. Duplex Compute is $0.01 per connected minute, or $0.60 per hour. With these inputs, the GPU beats $3.00 an hour above about 17% utilization (2 ÷ (4 × 0.167) ≈ $3.00). It only beats $0.60 an hour above about 83% utilization, before any of the extra costs listed above.
The same comparison by month: one GPU kept warm for 730 hours at $2 costs $1,460. To match that, you would need 487 conversation-hours at $3.00 per hour, or 2,433 conversation-hours at $0.60.
GPU cost only. Defaults are the hypothetical inputs above; replace sessions per GPU with your own load-test result.
When self-hosting is the right call
Cost is rarely the strongest reason. Audio that cannot leave your network is a strong one. A model you want to fine-tune on your own speakers is another. So is a steady, predictable load that keeps GPUs busy around the clock. Those teams should plan a load test before a budget: pick the serving path, run concurrent streams with real audio, and record the stream count at which latency or quality breaks.
Managed hosting of the orchestration layer has the same utilization problem on a smaller scale. Pipecat Cloud runs one active session per instance, and a larger instance profile does not add session slots. Warm reserved instances bill while idle.
If what you want is a full-duplex voice session with tools and longer conversations, and none of those reasons applies, a per-minute API moves the utilization risk to the provider. Duplex Compute is that kind of API, at $0.01 per connected minute. That rate does not include whatever backend your tools call. A cheap minute is only cheap if the task completes.
Related notes
- Nemotron VoiceChat tool calling
- Full-duplex voice AI vs. turn-based
- Voice agent pricing: calculator and billing formulas
- What does an hour of conversation cost?
- Voice agent evaluation
Weighing a GPU against a per-minute rate?
Tell us your expected voice hours, your peak concurrency, the tools your agent needs and what you run today. We'll tell you whether our full-duplex API covers it.
Request access