Chat and streaming

Pick a format

Streaming

Set stream to true. The OpenAI format sends data: lines and ends with data: [DONE], and the Anthropic format sends its usual events from message_start to message_stop. On the OpenAI format, add stream_options with include_usage to get the token counts and the charge in one last chunk.

stream = client.chat.completions.create(
    model="gemini-3.8-flash",
    messages=[{"role": "user", "content": "Write four lines about moths and lamps"}],
    stream=True,
    stream_options={"include_usage": True},
)
for chunk in stream:
    if chunk.choices:
        print(chunk.choices[0].delta.content or "", end="")
    elif chunk.usage:
        print("\n", chunk.usage)

A stream that fails part way ends with an error event instead, and on the OpenAI format it sends no [DONE]. You pay for the input and for the output that arrived before it stopped, and a stream that fails before any output costs nothing.

Tools

Tools work on both formats and are translated for each model's host. A model whose capabilities.tools is false refuses a request with tools, and one whose capabilities.forced_tool_choice is false refuses a forced choice, required or a named function on the OpenAI format and any or a named tool on the Anthropic format, so send auto to those.

Images in a prompt

Send an image as a URL or as base64 data in JPEG, PNG, GIF or WebP. Each image counts as 5,000 tokens in the hold, and you then pay what the model reports.

Good to know

• Each request asks for one reply, so n must be 1 when you send it.
• On Claude models response_format must be text, and on the GPT, Grok, DeepSeek and Kimi models it must be text or json_schema with a schema, since json_object is refused.
• A conversation longer than the model's context_length is refused with 400 before anything is held.
• Settings a model can't take are dropped or refused, as What is removed lists.