Skip to main content
Set stream: true and the gateway returns a text/event-stream response in the standard chat completion chunk format. The stock SDKs handle the stream for you.

The final chunk carries the cost

On streaming chat completions the gateway sets stream_options.include_usage, so the last data chunk before [DONE] has a usage object with the model’s metered cost for the call in dollars:
You do not need to set stream_options yourself, and setting it does no harm. The credits for a stream settle at this final chunk, based on that metered cost as described on pricing.

Disconnects

Closing the connection mid-stream does not cancel generation at the model. The call is still charged for what the model generated. Keep max_tokens bounded on user-facing streams.

Other streaming paths

POST /v1/completions, POST /v1/responses, and POST /v1/messages stream in their own native event formats (text completion chunks, Responses API events, and Anthropic-style events respectively). The Anthropic SDK’s client.messages.stream() works against /v1/messages; see Messages.