Responses API vs Chat Completions for Agents: Choosing a Contract

Responses API vs Chat Completions is mostly a question of who owns conversation state, not which endpoint looks newer. Chat Completions makes your orchestrator resend the whole messages array on every turn. Responses can continue from a stored response ID instead. We read TokenLab's published API docs and live model API on 2026-10-03, and the answer also depends on which formats your model accepts. So we start with a decision table.

Agent requirement	Pick	Why
Existing OpenAI-compatible chat client, portable across many models	Chat Completions (/v1/chat/completions)	Broadest model compatibility, per the API formats guide (observed 2026-10-03)
Continue a conversation from a stored server-side response	Responses (/v1/responses)	previous_response_id and store are Responses fields
Arbitrary history edits, pruning, or message masking between turns	Chat Completions	You own the full messages array
Background responses or Responses events	Responses	Documented for models that support them
Persistent WebSocket connection with continuation	Responses	wss://api.tokenlab.sh/v1/responses
Portable JSON-object output	Chat Completions	response_format: {"type": "json_object"}
Model does not list openai_responses	Chat Completions, or the model's native format	Responses is only for models that list it
Claude thinking blocks, Claude tool use, prompt-cache fields	Anthropic Messages (/v1/messages)	Chat Completions does not guarantee those fields
Gemini contents, files, cached content, Gemini tools	Gemini (/v1beta/models/:model:generateContent)	Keep Gemini-native fields on the Gemini route

Key Takeaways

Responses API vs Chat Completions: the same task, two requests

Take one single-turn task: explain why the sky is blue in two sentences. Here it is as a Chat Completions request, using the client setup from the API formats guide (observed 2026-10-03):

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["TOKENLAB_API_KEY"],
    base_url="https://api.tokenlab.sh/v1",
)

response = client.chat.completions.create(
    model="gpt-5.6-terra",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Explain why the sky is blue in two sentences."},
    ],
    max_tokens=1000,
)

print(response.choices[0].message.content)

And here is the same task as a Responses request:

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["TOKENLAB_API_KEY"],
    base_url="https://api.tokenlab.sh/v1",
)

response = client.responses.create(
    model="gpt-5.6-terra",
    instructions="You are a helpful assistant.",
    input="Explain why the sky is blue in two sentences.",
    max_output_tokens=1000,
)

print(response.output_text)

The fields map one to one for this simple case. The system message becomes top-level instructions. messages becomes input, which accepts a string or an array of items. max_tokens becomes max_output_tokens.

The response bodies differ more. Chat Completions returns choices[0].message and a finish_reason. Responses returns an output array of typed items and a status field. The Create Response reference (observed 2026-10-03) lists queued, in_progress, completed, incomplete, failed, and cancelled, and tells you to check status even when the HTTP request succeeds. Usage keys differ too: prompt_tokens and completion_tokens on one side, input_tokens and output_tokens on the other. The reference examples show 20/9/29 and 10/12/22 respectively. Those are sample payloads, not measurements.

Continuing a conversation

Imagine a support agent that has just answered one question and now gets "What about order #4472?" With Chat Completions, you append the new user message to the stored array and resend everything. With Responses, you pass the previous response's id and send only the new turn:

{
  "model": "gpt-5.6-terra",
  "previous_response_id": "resp_abc123",
  "input": "What about order #4472?"
}

store defaults to true, so the previous response is kept for later retrieval unless you turn it off. Your application still has to persist the returned id durably between hops. If you need custom pruning or message masking, the stateless format is the more direct fit.

A rough estimate shows why stateless resending matters at agent scale. Assume a 10,000-token history resent on each of 20 calls. That is 200,000 input tokens. At the gpt-5.6-terra input price of $0.60 per 1M tokens, that comes to 200,000 × 0.60 / 1,000,000 = $0.12 (estimate; assumed token counts, no caching). The price comes from the live model API, observed 2026-10-03, with pricing updated 2026-10-02T16:53:30.068Z.

Two caveats apply. The docs we read do not say whether server-held context is billed as input tokens, so compare usage on your own traffic. And requests above 272K input tokens are priced at 2x input and 1.5x output for the full request: $1.20 and $5.40 per 1M instead of $0.60 and $3.60. Long stateless histories can cross that line.

Tool calling: request and response shape

Both formats follow the loop in the Structured Outputs & Tool Calling guide (observed 2026-10-03). Send tools, read the call, run it in your own backend, return the result in the same format, and repeat until a final answer arrives. The shapes differ.

Chat Completions. Tools are declared as {"type": "function", "function": {...}}. The model answers with tool_calls on choices[0].message, and finish_reason is tool_calls. To continue, you resend the whole array, including the assistant message with its tool_calls and a role: "tool" message carrying tool_call_id. The values below are illustrative; lookup_customer is the tool name used in the guide:

{
  "model": "gpt-5.6-terra",
  "messages": [
    {"role": "user", "content": "Look up ana@example.com"},
    {
      "role": "assistant",
      "content": null,
      "tool_calls": [
        {
          "id": "call_xyz789",
          "type": "function",
          "function": {"name": "lookup_customer", "arguments": "{\"email\": \"ana@example.com\"}"}
        }
      ]
    },
    {"role": "tool", "tool_call_id": "call_xyz789", "content": "{\"status\": \"active\"}"}
  ]
}

Responses. The model answers with function_call items inside output, each carrying a call_id, name, and arguments. You return the result as a function_call_output input item and point at the previous response:

{
  "model": "gpt-5.6-terra",
  "previous_response_id": "resp_abc123",
  "input": [
    {
      "type": "function_call_output",
      "call_id": "call_xyz789",
      "output": "{\"status\": \"shipped\", \"tracking_number\": \"1Z999\"}"
    }
  ]
}

You never rebuild the intermediate assistant tool message. parallel_tool_calls exists in both formats. In the Create Response reference it defaults to true. The Chat reference lists it without a default and says it applies "when supported by the selected model."

We found one gap. The pages we read describe Responses tools and tool_choice only in general terms. Before you port tool schemas, read the tool definition shape in the Create Response reference and test it on your target model. Also keep these rules from the guide:

• TokenLab does not convert built-in tools between formats.
• Validate tool arguments before any side effect.
• Make tool execution idempotent, because a client retry can repeat the same call.
• Apply your own permission checks.

Streaming: events a client must handle

Chat Completions streams SSE chunks. Your client reads choices[0].delta.content, records finish_reason, and skips chunks with no choices. With stream_options={"include_usage": True}, the usage arrives in such a chunk. The Streaming guide (observed 2026-10-03) treats stop as complete, length as a token limit, and tool_calls as a request to run tools.

Responses streams named events. The guide's consumer handles these:

• response.output_text.delta for text to append
• response.completed for the success marker
• response.failed, response.incomplete, and error as terminal problems

import os
from openai import OpenAI

with OpenAI(
    api_key=os.environ["TOKENLAB_API_KEY"],
    base_url="https://api.tokenlab.sh/v1",
    timeout=30.0,
    max_retries=0,
) as client:
    completed = False
    with client.responses.create(
        model="gpt-5.6-terra",
        input="Write a short poem.",
        stream=True,
    ) as stream:
        for event in stream:
            if event.type == "response.output_text.delta":
                print(event.delta, end="", flush=True)
            elif event.type == "response.completed":
                completed = True
            elif event.type in {"response.failed", "response.incomplete", "error"}:
                raise RuntimeError(f"Stream ended with {event.type}")
    if not completed:
        raise RuntimeError("Stream closed before response.completed")

Three client rules follow from the guide:

1. After the first event arrives, an interrupted stream is incomplete and is not restarted automatically. Keep partial output if it helps, but do not mark it complete. A repeated request may differ and cost again.
2. Use the SDK to parse SSE, because one network chunk is not necessarily one complete event.
3. Cancel the upstream stream when the user stops or disconnects, and keep the key on your server.

Responses also offers a WebSocket at wss://api.tokenlab.sh/v1/responses. Responses there are always streamed, background and response.cancel are unsupported, each connection runs one active response at a time, and a connection lasts up to 60 minutes. Events carry a response-scoped sequence_number. Stream events named error are flat objects, while connection and protocol failures use a nested error object, so handle both shapes.

Check which formats a model accepts before you send

Read tokenlab.accepted_request_formats from the model page or from GET /v1/models/{model}, as described in the Get a Model reference (observed 2026-10-03):

curl "https://api.tokenlab.sh/v1/models/gpt-5.6-terra"

The response includes a tokenlab object with accepted_request_formats, pricing, limits, and lifecycle. The values map to endpoints as the errors guide describes: openai_chat_completions to /v1/chat/completions, openai_responses to /v1/responses, anthropic_messages to /v1/messages, and gemini_generate_content to /v1beta/models/{model}:generateContent. An accepted format confirms the endpoint only. Individual tools and fields can still vary by model.

We checked the three models this article references against the live model API on 2026-10-03:

Model	Accepted formats	Max input / output tokens	Price per 1M in / out (USD)	Source
gpt-5.6-terra	openai_chat_completions, openai_responses	1,050,000 / 128,000	0.60 / 3.60 (inputs up to 272K)	live API
claude-sonnet-5	anthropic_messages, openai_chat_completions	1,000,000 / 128,000	0.90 / 4.50	live API
gemini-3.5-flash	gemini_generate_content, openai_chat_completions	1,048,576 / 65,536	0.75 / 4.50	live API

All three were in the active lifecycle stage with no replacement model, and pricing was updated 2026-10-02T16:53:30.068Z. So only gpt-5.6-terra takes a Responses request today. The other two work through Chat Completions or their native routes. Re-check before you ship, since lifecycle and prices can change.

Error handling also matters here. A misspelled model on a create call returns 400 model_not_found and may include did_you_mean. Asking GET /v1/models/{model} about an unknown ID returns 404 model_not_found. A 503 all_channels_failed with retryable: false means that repeating the request will not help. Do not swap models automatically unless your product has permission to do so.

Migration checklist: Responses API vs Chat Completions

Use this order when you move an agent from Chat Completions to Responses.

1. Confirm the model lists openai_responses. If it does not, stop here. Staying on Chat Completions is the supported path.
2. Pick a cutover boundary. Start new conversations on Responses. Do not switch a live thread mid-flight, because the formats represent state and tool calls differently.
3. Convert stored history deliberately. If you must migrate old threads, convert and validate them in your application before the next request.
4. Rename the core fields. messages becomes input, system or developer messages become instructions, and max_tokens or max_completion_tokens becomes max_output_tokens.
5. Rewrite the tool loop. Replace choices[0].message.tool_calls parsing with function_call items, and replace role: "tool" messages with function_call_output items keyed by call_id.
6. Persist response IDs. Store each id so you can pass previous_response_id. Decide on store and retention up front, and check how long TokenLab keeps stored responses, because the pages we read do not say.
7. Rewrite the stream consumer. Handle response.output_text.delta, response.completed, response.failed, response.incomplete, and error. Treat a closed stream without response.completed as a failure.
8. Check status. An HTTP success can still carry incomplete or failed.
9. Audit parameters that may not carry over. The Create Response page lists temperature, top_p, tools, parallel_tool_calls, reasoning, and text. It does not list n, logprobs, logit_bias, frequency_penalty, presence_penalty, seed, top_k, stop, user, or stream_options. Verify each one on your model before relying on it. TokenLab may pass unknown fields through, but that does not mean the model accepts them.
10. Map JSON output. Chat uses response_format. Responses uses the text object, and text.format support depends on the model.
11. Plan around missing endpoints. Cancel, input-items, and input-tokens endpoints are not currently available. Deleting a stored response does not cancel generation.
12. Replay-test with retries off. Keep max_retries=0 and decide in your own code when a retry is safe. Timed-out generations should not replay automatically.

FAQ

Can I send a Responses request to Claude Sonnet 5 or Gemini 3.5 Flash?

Not according to the live model API on 2026-10-03. claude-sonnet-5 lists anthropic_messages and openai_chat_completions. gemini-3.5-flash lists gemini_generate_content and openai_chat_completions. Use Chat Completions or the native route, and check accepted_request_formats again before you ship.

Do I stop paying for earlier turns when I use previous_response_id?

The pages we read do not say. They document that the server can continue from a stored response, but not how restored context is billed. Run a few multi-turn calls and compare the usage fields against your stateless baseline before you plan on savings.

Can I switch formats in the middle of a conversation?

No. TokenLab's guides say to keep one format per conversation, because each format represents state and tool calls differently. If you must migrate stored history, convert and validate it in your application before sending the next request.

How do I cancel a running Responses request?

A cancel endpoint is not currently available. Deleting a stored response does not cancel generation, and WebSocket responses do not support response.cancel. For streams, close the upstream stream from your server when the user stops or disconnects.

Before you change endpoints, open the TokenLab model catalog and confirm accepted_request_formats for the exact model your agent calls.