OpenAI Realtime API function calling: from a spoken request to a finished action

The model only proposes a call. Your application validates it, runs it, returns the result under the same call_id, and asks for the next response. This note walks the loop for gpt-realtime-2.1 and covers the parts the docs leave to you.

Duplex Compute · October 13, 2026 · Rates and versions checked October 13, 2026

The loop in four events

The Realtime conversations guide describes function calling as four steps. You list functions. The model adds a function call item with arguments. Your client runs its own code. Your client adds the output and asks the model to respond. Nothing on OpenAI's side executes your function.

1. Advertise the tools

Tools can live on the session or on a single response: session.tools in session.update, or response.tools in response.create. The guide's example uses the session form with "tool_choice": "auto". Here are two tools for a restaurant booking agent, one that reads and one that writes.

{
  "type": "session.update",
  "session": {
    "type": "realtime",
    "model": "gpt-realtime-2.1",
    "instructions": "Confirm the party size, date and time before booking. Only say a table is booked after book_table returns status ok.",
    "tools": [
      {
        "type": "function",
        "name": "check_availability",
        "description": "Check whether a table is free. Does not book anything.",
        "parameters": {
          "type": "object",
          "properties": {
            "date": { "type": "string", "description": "YYYY-MM-DD" },
            "time": { "type": "string", "description": "HH:MM, 24-hour" },
            "party_size": { "type": "integer", "minimum": 1, "maximum": 12 }
          },
          "required": ["date", "time", "party_size"]
        }
      },
      {
        "type": "function",
        "name": "book_table",
        "description": "Create or update this caller's single reservation.",
        "parameters": {
          "type": "object",
          "properties": {
            "date": { "type": "string" },
            "time": { "type": "string" },
            "party_size": { "type": "integer", "minimum": 1, "maximum": 12 }
          },
          "required": ["date", "time", "party_size"]
        }
      }
    ],
    "tool_choice": "auto"
  }
}

The descriptions steer the model's choice of tool and arguments. They enforce nothing. A maximum of 12 in the schema does not stop a call with 40; your handler has to check.

2. Receive the call

When the model decides to call a function, its response contains a function_call item in place of speech. The guide shows it inside response.done:

{
  "type": "response.done",
  "response": {
    "status": "completed",
    "output": [
      {
        "type": "function_call",
        "name": "book_table",
        "call_id": "call_sHlR7iaFwQ2YQOqm",
        "arguments": "{\"date\":\"2026-10-16\",\"time\":\"19:30\",\"party_size\":6}"
      }
    ]
  }
}

arguments is a string, so parse it yourself. Arguments also stream as response.function_call_arguments.delta, and the server events reference lists response.function_call_arguments.done for when they finish. Reading response.done is simpler because one response can carry more than one call. Keep the call_id. The guide calls it the ID you will need to pass a function call result back to the model.

3. Validate, then execute

Treat the arguments as user input that went through a speech recognizer and a language model. Check types, ranges and dates. Check that this caller may make this booking. Return a failed check as a normal output so the model can ask the caller to fix it.

Read-only tools such as check_availability can run as soon as they arrive. Writes need two more guards, covered below: idempotency and a timeout that does not lie.

4. Return the output and ask for a response

{
  "type": "conversation.item.create",
  "item": {
    "type": "function_call_output",
    "call_id": "call_sHlR7iaFwQ2YQOqm",
    "output": "{\"status\":\"ok\",\"reservation_id\":\"R-2041\",\"party_size\":6}"
  }
}

{ "type": "response.create" }

The output is a string, usually serialized JSON. Adding it does not make the model speak. The guide sends response.create afterward to get a reply built from the result. If the response held several calls, add every output first and send one response.create.

A sketch of the handler

This is a sketch, not tested code. It assumes a Node WebSocket ws already connected to the Realtime API, a db with an upsert, and a sessionId you assign per conversation.

const outputs = new Map();   // call_id -> output string, so a repeated call_id never runs twice
let responseActive = false;

const send = (event) => ws.send(JSON.stringify(event));
const result = (status, detail) => JSON.stringify({ status, detail });

ws.on("message", async (raw) => {
  const event = JSON.parse(raw);
  if (event.type === "response.created") responseActive = true;
  if (event.type !== "response.done") return;
  responseActive = false;

  const calls = event.response.output.filter((item) => item.type === "function_call");
  if (calls.length === 0) return;

  for (const call of calls) {
    if (!outputs.has(call.call_id)) outputs.set(call.call_id, await runTool(call));
    send({
      type: "conversation.item.create",
      item: { type: "function_call_output", call_id: call.call_id, output: outputs.get(call.call_id) },
    });
  }
  // If the caller started talking while the tool ran, VAD may already have started a response.
  if (!responseActive) send({ type: "response.create" });
});

async function runTool(call) {
  let args;
  try { args = JSON.parse(call.arguments); } catch { return result("invalid_arguments", "not JSON"); }
  const tool = TOOLS[call.name];
  if (!tool) return result("unknown_tool", call.name);
  const problem = tool.validate(args);
  if (problem) return result("invalid_arguments", problem);
  try {
    return result("ok", await withTimeout(tool.run(args), tool.timeoutMs));
  } catch (err) {
    if (err.name === "Timeout") return result("unknown", "Still running. Do not tell the caller it worked.");
    return result("failed", err.message);
  }
}

const TOOLS = {
  book_table: {
    timeoutMs: 8000,
    validate: (a) => (Number.isInteger(a.party_size) && a.party_size >= 1 && a.party_size <= 12
      ? null : "party_size must be 1 to 12"),
    // One reservation per conversation: a later call changes it instead of adding a second one.
    run: (a) => db.upsertReservation({ key: sessionId, ...a }),
  },
  // check_availability: same shape, no idempotency key needed.
};

"Book six, actually seven"

The caller says "a table for six on Friday at 7:30" and, a second later, "actually, seven". The model may call book_table with six before the correction lands. It may then call it again with seven. Those are two calls with two different call_id values, so deduplicating by call_id does not help.

The fix belongs in your data model. Key the write on the task, here one reservation per conversation, so a second call updates the first. Keep the call_id map too: it stops a retried or replayed call from running twice. The instruction to confirm details first, which OpenAI's own session.update example includes, reduces how often the six-person call fires at all. It does not replace the key.

Timeouts and failures

A voice caller hears silence while your handler runs. Put a deadline on every tool. When it expires, return a result that says the outcome is unknown and keep tracking the job. Do not return failed for a write that may still succeed: the model will say it failed, the caller will ask again, and you can book twice.

Return failures as structured output with a status the model can act on: invalid_arguments means ask the caller, failed means apologize and offer another path, unknown means say you are still checking. When the slow job finishes, add the real outcome to the conversation and trigger a new response.

Speak only from the outcome

The model's next sentence is built from whatever you put in output. If the output says ok, it will confirm. Write what your system actually did: the row you saved, with its final values. Include the fields the caller needs to hear, such as the final party size and a reference number, and leave out what you never want read aloud.

Test the two things separately. OpenAI's voice agent evaluation guide checks the tool calls and the resulting application state, not only the transcript. A correct sentence over a wrong database row is a failed task.

Interruptions while a tool runs

With VAD on, the Realtime API detects user speech, cancels the ongoing response and starts a new one. That cancels speech. It does not reach into your process and stop book_table. Decide per tool whether a running call should finish, be cancelled, or be superseded by the next call. The upsert above picks "superseded".

On a WebSocket you also own playback: stop local audio and send conversation.item.truncate for the unplayed part. The interruption note covers that path.

If you want the voice layer to keep talking with the caller while a long tool runs, that is a different architecture. GPT-Live delegates to a backend, and its delegation guide says interrupting the spoken conversation leaves backend work running. Duplex Compute is building a full-duplex voice API with tool calling at $0.01 per connected minute for the same kind of application.

The same problem in Pipecat

Pipecat makes interruption behavior a per-tool setting. Its function calling docs say a direct function is cancelled when the user interrupts by default. Decorating it with @tool_options(cancel_on_interruption=False, timeout_secs=30) makes the call asynchronous: the conversation is not held while it runs, and a result that arrives after the conversation moved on is injected as a developer message that triggers a new inference.

@tool_options(cancel_on_interruption=False, timeout_secs=30)
async def book_table(params: FunctionCallParams, date: str, time: str, party_size: int):
    """Create or update this caller's single reservation."""
    ...

The same docs note that a timed-out handler is thrown asyncio.CancelledError, and that work it spawned into its own task is not cancelled with it. Cancelling a handler does not undo a write. The idempotency key still matters.

Related notes

• Why your Realtime agent keeps talking after an interruption
• GPT-Live vs. Realtime API
• Low-cost voice APIs with tool calling
• Voice agent evaluation