Skip to main content
Set stream: true and the gateway returns a text/event-stream response in the standard chat completion chunk format. The stock SDKs handle the stream for you.

The final chunk carries usage

On streaming chat completions, the last data chunk before [DONE] has a usage object with the call’s token counts. You do not need to set stream_options yourself.
The credits for a stream settle when it ends. Headers are sent before the stream starts, so a stream carries x-predev-request-id but not x-predev-credits-charged. To read the charge, pass the chunk’s id to GET /v1/generation, or see the totals in GET /v1/usage. Lines that start with : are keep-alive comments; SSE clients ignore them.

Disconnects

If your client disconnects mid-stream, generation stops shortly after, and the call is charged for what the model generated. Keep max_tokens bounded on user-facing streams.

Other streaming paths

POST /v1/completions, POST /v1/responses, and POST /v1/messages stream in their own native event formats (text completion chunks, Responses API events, and Anthropic-style events). The Anthropic SDK’s client.messages.stream() works against /v1/messages; see Messages.