Build an AI voice tutor that hears the student's correction

A voice tutor for a learning app is a small loop: ask, wait, listen, judge, record. The voice model runs the conversation. Your application owns the lesson, the answer key and the record of what the student got right.

Duplex Compute · October 13, 2026 · Rates and versions checked October 13, 2026

This guide is for developers adding spoken practice to a learning product. It covers architecture and cost. It makes no claim that voice tutoring improves learning; that needs your own study with your own students.

Keep the loop narrow

One session works on one lesson objective. The tutor fetches the next item, asks it, waits for the student to finish thinking, judges the answer against the key, explains if needed and records the attempt. Then it repeats. Free conversation can sit around that loop, but the loop is what you test.

start session (student_id, lesson_id)
  -> get_lesson_items      your server returns items and answer keys
  loop:
     ask item              voice model speaks
     wait                  long pauses are normal; do not cut in
     hear answer           student may self-correct: "x equals four, no, five"
     judge                 against the key from your server, not model memory
     explain or move on
     -> record_attempt     your server writes progress
  end: summary            your server stores it for next time

Google's Live API best practices give the same advice for any voice agent. Write the persona, then the conversational rules in order, then guardrails. Name each tool inside the step that uses it. They also note that "the model performs best on tasks with single function calls," which suits a loop with one lookup and one write per item.

Pick the architecture

OpenAI's voice agents guide describes three shapes, and each one changes what a tutor can do.

Language practice leans toward speech-to-speech, because pronunciation is the point. Maths and recall practice can grade a transcript. Whatever you pick, keep the answer key on your server and send it through a tool. A key in the tool result gives the model something to check the answer against, and you can audit it later.

Two tools do most of the work

The tutor needs one read and one write. Write each description with the condition for calling it, as Google's tool examples do.

[
  {
    "name": "get_lesson_items",
    "description": "Returns the next practice items for this lesson with their answer keys. Call once at the start of the session, and again only when all returned items are done.",
    "parameters": {
      "type": "object",
      "properties": {
        "student_id": { "type": "string" },
        "lesson_id": { "type": "string" },
        "count": { "type": "integer", "description": "Items to return, 1 to 10." }
      },
      "required": ["student_id", "lesson_id"]
    }
  },
  {
    "name": "record_attempt",
    "description": "Saves the student's final answer to one item. Call once per item, only after the student has finished answering, including any self-correction. Never call it for an answer the student withdrew.",
    "parameters": {
      "type": "object",
      "properties": {
        "attempt_id": { "type": "string", "description": "Unique per item per session, for example session_id + item_id." },
        "item_id": { "type": "string" },
        "final_answer": { "type": "string" },
        "correct": { "type": "boolean" },
        "hints_used": { "type": "integer" }
      },
      "required": ["attempt_id", "item_id", "final_answer", "correct"]
    }
  }
]

The attempt_id makes the write idempotent. If the session drops and the model calls record_attempt again after reconnecting, your server sees the same key and keeps one row. The model cannot guarantee that. Your handler can.

The student's self-correction is the case to test first. "x equals four, no, five" must produce one attempt with final_answer 5. The voice agent evaluation note has a scenario template for exactly this kind of corrected request.

Let the student think

Students often stop mid-answer to think. Silence-based turn detection reads a long pause as the end of a turn and answers over the student. Both major APIs have a setting for this. In OpenAI's Realtime API, semantic_vad waits longer when the words sound unfinished, and eagerness: "low" "will let the user take their time to speak." In Gemini Live, the END_SENSITIVITY_LOW setting makes automatic detection end speech less often.

Tune it with recordings of real students answering hard items. A tutor that waits a beat too long is a mild annoyance. One that keeps cutting in teaches students to rush.

Interruptions and writes

Students interrupt explanations: "wait, I get it." Stop playback at once. Google's best practices say to discard the client audio buffer as soon as the server reports interrupted. Then let the student talk.

An interruption should not cancel a progress write. In Pipecat, a tool's cancel_on_interruption defaults to true, which cancels the call when the user interrupts. Set it to false for record_attempt, so the write completes while the student speaks. Leave lookups cancellable if you like; a fresh lookup is cheap.

Keep progress outside the session

The voice session forgets. Gemini Live's session guide limits an audio-only session to 15 minutes without context compression, and compression extends it by dropping the oldest turns. Connections reset after about 10 minutes and need session resumption. A resumption handle expires two hours after the session ends. None of this is a learner record.

Put everything you need next week in your database as it happens: attempts, the items the student found hard, the hint level that worked. Start each session by loading a short summary into the instructions. After a dropped connection, the tutor can then say where you were from your data, not from what the model kept. The session resumption note walks through the reconnect itself.

What a 30-minute session costs

The example below is arithmetic on published rates, not a measured bill. It assumes a 30-minute connected session in which the tutor speaks for 12 minutes and the student for 10, with 8 minutes of thinking and silence.

The Gemini figure is a floor: 30 minutes of listening at $0.005 is $0.15, and 12 minutes of speech at $0.018 is $0.216. Gemini bills the whole retained conversation again on every turn. A tutoring loop has many short turns, so that line grows quickly. The Gemini Live pricing note works a 30-minute example where compression settings move the audio total between about $1.50 and $2.80.

Scale it before you commit. A product with 1,000 students doing four 30-minute sessions a month runs 2,000 connected hours. That is $6,000 a month in GPT-Live voice time before backend costs, or $1,200 at Duplex's voice-session rate. Neither figure includes the model that judges answers if it runs separately, your servers or support. The voice agent pricing hub has the formulas for the full bill.

Related notes

• Semantic VAD vs. silence-based turn detection
• Low-cost voice APIs with tool calling
• Gemini Live session resumption
• Voice agent evaluation
• Voice agent pricing: calculator and billing formulas