Skip to main content

AI Gateway

Streaming

Stream chat completions over SSE, read token usage from the final chunk, and look up the charge afterward.

Set stream: true and the gateway returns a text/event-stream response in the standard chat completion chunk format. The stock SDKs handle the stream for you.

curl --fail-with-body -N https://api.pre.dev/v1/chat/completions \
  -H "Authorization: Bearer $PREDEV_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "google/gemini-3.8-flash",
    "stream": true,
    "messages": [{ "role": "user", "content": "Count from one to ten." }]
  }'
import OpenAI from 'openai';

const client = new OpenAI({ baseURL: 'https://api.pre.dev/v1', apiKey: process.env.PREDEV_API_KEY! });

const stream = await client.chat.completions.create({
  model: 'google/gemini-3.8-flash',
  stream: true,
  messages: [{ role: 'user', content: 'Count from one to ten.' }],
});
for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? '');
  if (chunk.usage) console.log('\ntokens:', chunk.usage.total_tokens, 'generation:', chunk.id);
}
import os
from openai import OpenAI

client = OpenAI(base_url="https://api.pre.dev/v1", api_key=os.environ["PREDEV_API_KEY"])

stream = client.chat.completions.create(
    model="google/gemini-3.8-flash",
    stream=True,
    messages=[{"role": "user", "content": "Count from one to ten."}],
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)
    if chunk.usage:
        print("\ntokens:", chunk.usage.total_tokens, "generation:", chunk.id)
import { createOpenAI } from '@ai-sdk/openai';
import { streamText } from 'ai';

const predev = createOpenAI({ baseURL: 'https://api.pre.dev/v1', apiKey: process.env.PREDEV_API_KEY! });

const result = streamText({
  model: predev('google/gemini-3.8-flash'),
  prompt: 'Count from one to ten.',
});
for await (const part of result.textStream) process.stdout.write(part);

The final chunk carries usage

On streaming chat completions, the last data chunk before [DONE] has a usage object with the call’s token counts. You do not need to set stream_options yourself.

data: {"id":"pdg-7c1e4b52-9a3d-4f8e-b0c6-2d5a8e913f47","choices":[],"usage":{"prompt_tokens":12,"completion_tokens":31,"total_tokens":43}}

data: [DONE]

The credits for a stream settle when it ends. Headers are sent before the stream starts, so a stream carries x-predev-request-id but not x-predev-credits-charged. To read the charge, pass the chunk’s id to GET /v1/generation, or see the totals in GET /v1/usage.

Lines that start with : are keep-alive comments; SSE clients ignore them.

Disconnects

If your client disconnects mid-stream, generation stops shortly after, and the call is charged for what the model generated. Keep max_tokens bounded on user-facing streams.

Other streaming paths

POST /v1/completions, POST /v1/responses, and POST /v1/messages stream in their own native event formats (text completion chunks, Responses API events, and Anthropic-style events). The Anthropic SDK’s client.messages.stream() works against /v1/messages; see Messages.

Was this page helpful?

On this page