Create Chat Completion

ID of the model to use. See Models for available options.

A list of messages comprising the conversation.

Each message object contains:

• role (string): system, developer, user, assistant, tool, function
• content (string | array | null): The message content

For an assistant message containing tool_calls, content may be omitted or set to null.

When content is an array, TokenLab supports structured multimodal blocks for compatible models:

• text: { "type": "text", "text": "..." }
• image: { "type": "image_url", "image_url": { "url": "https://..." } }
• video: { "type": "video_url", "video_url": { "url": "https://..." } }
• audio: { "type": "audio_url", "audio_url": { "url": "https://..." } }

For multimodal input, use publicly reachable https URLs. Supported media types depend on the selected model.

Sampling temperature. Support, allowed values, and the default depend on the selected model; omit this field to use its default.

Maximum number of tokens to generate.

For deepseek-v4.1-flash (and the deepseek-flash alias), the model defaults to thinking first. Thinking tokens count against max_tokens. If max_tokens is too small, the whole budget can go to thinking and leave an empty message body while the request is still billed. For long answers, prefer max_tokens around 32000, or turn thinking off. To turn thinking off, add "thinking": {"type": "disabled"} (or "reasoning_effort": "none") to the request body; this works for both streaming and non-streaming requests. "enable_thinking": false and "reasoning": {"enabled": false} do not turn it off.

If true, partial message deltas will be sent as SSE events.

For long outputs (thousands of tokens or more), set stream=true so non-streaming clients do not time out waiting for the full response. Team measurement: GPT-6.1 Sol producing about 12k characters took about 147 seconds without streaming.

Options for streaming. Set include_usage: true to receive token usage in stream chunks.

Nucleus sampling parameter. We recommend altering this or temperature, not both.

Number between -2.0 and 2.0. Positive values penalize repeated tokens.

Number between -2.0 and 2.0. Positive values penalize tokens already in the text.

Stop sequence or list of sequences. Support and sequence limits depend on the selected model.

A list of tools the model may call (function calling).

Controls how the model uses tools. Options: auto, none, required, or a specific tool object.

Allow multiple tool calls in one assistant turn, when supported by the selected model.

Maximum tokens for the completion. Alternative to max_tokens, useful for newer reasoning-enabled model families.

Reasoning effort for models that support it. Accepted values depend on the selected model.

For Claude models, the value sets the model's thinking: an effort level on models with adaptive thinking, or a thinking budget on models that use one. none turns thinking off on models that allow it. While thinking is on, Claude rejects a temperature other than 1, top_k, a top_p outside 0.95–1, and a prefilled assistant message. A forced tool_choice works with adaptive thinking but not with a thinking budget.

Options for Claude models.

• prompt_caching (boolean): Set true to cache the start of the prompt. TokenLab marks the end of the system prompt and the latest user message as cache points. A later request that starts with the same content reads it from the cache. If any message already has cache_control, TokenLab adds no cache points.
• prompt_caching_ttl (string): 5m (default) or 1h. If the serving route does not support one hour, TokenLab uses five minutes and does not reject the request.

extra.prompt_caching and extra.prompt_caching_ttl are accepted as the same options. See Prompt caching for prices and how to check the cached tokens.

Sampling seed for models that support it. Identical output is not guaranteed.

Number of completions to generate (1-128).

Whether to return log probabilities.

Number of top log probabilities to return (0-20). Requires logprobs: true.

Top-K sampling for models that support it.

Response format. Use {"type": "json_object"} for JSON mode or {"type": "json_schema", "json_schema": {...}} for a JSON schema. Support depends on the selected model.

Modify the likelihood of specified tokens appearing. Map token IDs (as strings) to bias values from -100 to 100.

A unique identifier representing your end-user for abuse monitoring.

Unique identifier for the completion.

Always chat.completion.

Unix timestamp of when the completion was created.

The model used for completion.

List of completion choices.

Each choice contains:

• index (integer): Index of the choice
• message (object): The generated message
• finish_reason (string): Why the model stopped, for example stop, length, or tool_calls

When the model thinks, message can also contain:

• reasoning_content (string): The thinking text. Some models return only a summary, or no text.
• reasoning_details (array): Thinking blocks with their signatures. To continue a conversation with thinking, for example after a tool call, send them back unchanged in the assistant message.

In a stream, the same fields arrive as delta.reasoning_content and delta.reasoning_details.

Token usage statistics.

• prompt_tokens (integer): Tokens in the prompt
• completion_tokens (integer): Tokens in the completion, including thinking tokens
• total_tokens (integer): Total tokens used
• completion_tokens_details.reasoning_tokens (integer): Thinking tokens, when the model reports them