Skip to content

流式输出(SSE) ​

请求里加上 "stream": true,接口会以 SSE(Server-Sent Events)的形式边生成边返回,每一行 data: 是一小段内容。聊天界面、长文本生成都建议开启,用户能更早看到第一个字。

OpenAI 兼容接口 ​

bash
curl -N https://api.niceai.dev/v1/chat/completions \
  -H "Authorization: Bearer $NICEAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<模型名,见模型广场>",
    "stream": true,
    "stream_options": {"include_usage": true},
    "messages": [{"role": "user", "content": "写一首四行短诗"}]
  }'
python
import os
from openai import OpenAI

client = OpenAI(base_url="https://api.niceai.dev/v1", api_key=os.environ["NICEAI_API_KEY"])

stream = client.chat.completions.create(
    model="<模型名,见模型广场>",
    messages=[{"role": "user", "content": "写一首四行短诗"}],
    stream=True,
    stream_options={"include_usage": True},
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)
    if chunk.usage:
        print("\n", chunk.usage)
js
import OpenAI from 'openai'

const client = new OpenAI({ baseURL: 'https://api.niceai.dev/v1', apiKey: process.env.NICEAI_API_KEY })

const stream = await client.chat.completions.create({
  model: '<模型名,见模型广场>',
  messages: [{ role: 'user', content: '写一首四行短诗' }],
  stream: true,
  stream_options: { include_usage: true },
})
for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? '')
  if (chunk.usage) console.log('\n', chunk.usage)
}

返回的原始数据长这样,最后以 data: [DONE] 结束:

text
data: {"choices":[{"index":0,"delta":{"role":"assistant","content":""}}]}

data: {"choices":[{"index":0,"delta":{"content":"春风"}}]}

data: {"choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: [DONE]

流式下拿 token 用量

OpenAI 格式默认不在流里返回 usage。加上 "stream_options": {"include_usage": true},最后一个数据块会带上用量,此时它的 choices 是空数组,代码里要判空。

Anthropic Messages 接口 ​

Anthropic 的流用 event: 区分事件类型,文字增量在 content_block_delta 里:

python
import os
import anthropic

client = anthropic.Anthropic(base_url="https://api.niceai.dev", api_key=os.environ["NICEAI_API_KEY"])

with client.messages.stream(
    model="<模型名,见模型广场>",
    max_tokens=1024,
    messages=[{"role": "user", "content": "写一首四行短诗"}],
) as stream:
    for text in stream.text_stream:
        print(text, end="", flush=True)
    print("\n", stream.get_final_message().usage)
ts
import Anthropic from '@anthropic-ai/sdk'

const client = new Anthropic({ baseURL: 'https://api.niceai.dev', apiKey: process.env.NICEAI_API_KEY })

const stream = client.messages.stream({
  model: '<模型名,见模型广场>',
  max_tokens: 1024,
  messages: [{ role: 'user', content: '写一首四行短诗' }],
})
stream.on('text', (t) => process.stdout.write(t))
const final = await stream.finalMessage()
console.log('\n', final.usage)

事件顺序一般是:message_start → content_block_start → 多个 content_block_delta → content_block_stop → message_delta(含输出用量和 stop_reason)→ message_stop。中间可能穿插 ping,忽略即可。

排查 ​

现象可能原因与处理
一次性返回全部内容,没有逐字效果检查是否真的传了 stream: true;中间如果有 Nginx 等反向代理,要关闭缓冲(proxy_buffering off)
用 curl 看不到逐行输出加 -N 关闭 curl 缓冲
流到一半断开客户端或代理的读取超时太短。长输出、推理模型首字较慢,读取超时建议放宽到几分钟
解析报 JSON 错误每次读取的数据块不一定是完整的一行,要按换行符拼好再解析;以 : 开头的行是注释,跳过
拿不到 usageOpenAI 格式需加 stream_options.include_usage;Anthropic 格式在 message_start 和 message_delta 里

下一步 ​

NiceAI · 稳定的 AI 大模型 API 中转站