AI Gateway
Streaming
Stream chat completions over SSE, read token usage from the final chunk, and look up the charge afterward.
Set stream: true and the gateway returns a text/event-stream response in the standard chat completion chunk format. The stock SDKs handle the stream for you.
curl --fail-with-body -N https://api.pre.dev/v1/chat/completions \
-H "Authorization: Bearer $PREDEV_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "google/gemini-3.8-flash",
"stream": true,
"messages": [{ "role": "user", "content": "Count from one to ten." }]
}'import OpenAI from 'openai';
const client = new OpenAI({ baseURL: 'https://api.pre.dev/v1', apiKey: process.env.PREDEV_API_KEY! });
const stream = await client.chat.completions.create({
model: 'google/gemini-3.8-flash',
stream: true,
messages: [{ role: 'user', content: 'Count from one to ten.' }],
});
for await (const chunk of stream) {
process.stdout.write(chunk.choices[0]?.delta?.content ?? '');
if (chunk.usage) console.log('\ntokens:', chunk.usage.total_tokens, 'generation:', chunk.id);
}import os
from openai import OpenAI
client = OpenAI(base_url="https://api.pre.dev/v1", api_key=os.environ["PREDEV_API_KEY"])
stream = client.chat.completions.create(
model="google/gemini-3.8-flash",
stream=True,
messages=[{"role": "user", "content": "Count from one to ten."}],
)
for chunk in stream:
if chunk.choices and chunk.choices[0].delta.content:
print(chunk.choices[0].delta.content, end="", flush=True)
if chunk.usage:
print("\ntokens:", chunk.usage.total_tokens, "generation:", chunk.id)import { createOpenAI } from '@ai-sdk/openai';
import { streamText } from 'ai';
const predev = createOpenAI({ baseURL: 'https://api.pre.dev/v1', apiKey: process.env.PREDEV_API_KEY! });
const result = streamText({
model: predev('google/gemini-3.8-flash'),
prompt: 'Count from one to ten.',
});
for await (const part of result.textStream) process.stdout.write(part);The final chunk carries usage
On streaming chat completions, the last data chunk before [DONE] has a usage object with the call’s token counts. You do not need to set stream_options yourself.
data: {"id":"pdg-7c1e4b52-9a3d-4f8e-b0c6-2d5a8e913f47","choices":[],"usage":{"prompt_tokens":12,"completion_tokens":31,"total_tokens":43}}
data: [DONE]The credits for a stream settle when it ends. Headers are sent before the stream starts, so a stream carries x-predev-request-id but not x-predev-credits-charged. To read the charge, pass the chunk’s id to GET /v1/generation, or see the totals in GET /v1/usage.
Lines that start with : are keep-alive comments; SSE clients ignore them.
Disconnects
If your client disconnects mid-stream, generation stops shortly after, and the call is charged for what the model generated. Keep max_tokens bounded on user-facing streams.
Other streaming paths
POST /v1/completions, POST /v1/responses, and POST /v1/messages stream in their own native event formats (text completion chunks, Responses API events, and Anthropic-style events). The Anthropic SDK’s client.messages.stream() works against /v1/messages; see Messages.