外观
流式输出(SSE)
请求里加上 "stream": true,接口会以 SSE(Server-Sent Events)的形式边生成边返回,每一行 data: 是一小段内容。聊天界面、长文本生成都建议开启,用户能更早看到第一个字。
OpenAI 兼容接口
bash
curl -N https://api.niceai.dev/v1/chat/completions \
-H "Authorization: Bearer $NICEAI_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "<模型名,见模型广场>",
"stream": true,
"stream_options": {"include_usage": true},
"messages": [{"role": "user", "content": "写一首四行短诗"}]
}'python
import os
from openai import OpenAI
client = OpenAI(base_url="https://api.niceai.dev/v1", api_key=os.environ["NICEAI_API_KEY"])
stream = client.chat.completions.create(
model="<模型名,见模型广场>",
messages=[{"role": "user", "content": "写一首四行短诗"}],
stream=True,
stream_options={"include_usage": True},
)
for chunk in stream:
if chunk.choices and chunk.choices[0].delta.content:
print(chunk.choices[0].delta.content, end="", flush=True)
if chunk.usage:
print("\n", chunk.usage)js
import OpenAI from 'openai'
const client = new OpenAI({ baseURL: 'https://api.niceai.dev/v1', apiKey: process.env.NICEAI_API_KEY })
const stream = await client.chat.completions.create({
model: '<模型名,见模型广场>',
messages: [{ role: 'user', content: '写一首四行短诗' }],
stream: true,
stream_options: { include_usage: true },
})
for await (const chunk of stream) {
process.stdout.write(chunk.choices[0]?.delta?.content ?? '')
if (chunk.usage) console.log('\n', chunk.usage)
}返回的原始数据长这样,最后以 data: [DONE] 结束:
text
data: {"choices":[{"index":0,"delta":{"role":"assistant","content":""}}]}
data: {"choices":[{"index":0,"delta":{"content":"春风"}}]}
data: {"choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}
data: [DONE]流式下拿 token 用量
OpenAI 格式默认不在流里返回 usage。加上 "stream_options": {"include_usage": true},最后一个数据块会带上用量,此时它的 choices 是空数组,代码里要判空。
Anthropic Messages 接口
Anthropic 的流用 event: 区分事件类型,文字增量在 content_block_delta 里:
python
import os
import anthropic
client = anthropic.Anthropic(base_url="https://api.niceai.dev", api_key=os.environ["NICEAI_API_KEY"])
with client.messages.stream(
model="<模型名,见模型广场>",
max_tokens=1024,
messages=[{"role": "user", "content": "写一首四行短诗"}],
) as stream:
for text in stream.text_stream:
print(text, end="", flush=True)
print("\n", stream.get_final_message().usage)ts
import Anthropic from '@anthropic-ai/sdk'
const client = new Anthropic({ baseURL: 'https://api.niceai.dev', apiKey: process.env.NICEAI_API_KEY })
const stream = client.messages.stream({
model: '<模型名,见模型广场>',
max_tokens: 1024,
messages: [{ role: 'user', content: '写一首四行短诗' }],
})
stream.on('text', (t) => process.stdout.write(t))
const final = await stream.finalMessage()
console.log('\n', final.usage)事件顺序一般是:message_start → content_block_start → 多个 content_block_delta → content_block_stop → message_delta(含输出用量和 stop_reason)→ message_stop。中间可能穿插 ping,忽略即可。
排查
| 现象 | 可能原因与处理 |
|---|---|
| 一次性返回全部内容,没有逐字效果 | 检查是否真的传了 stream: true;中间如果有 Nginx 等反向代理,要关闭缓冲(proxy_buffering off) |
| 用 curl 看不到逐行输出 | 加 -N 关闭 curl 缓冲 |
| 流到一半断开 | 客户端或代理的读取超时太短。长输出、推理模型首字较慢,读取超时建议放宽到几分钟 |
| 解析报 JSON 错误 | 每次读取的数据块不一定是完整的一行,要按换行符拼好再解析;以 : 开头的行是注释,跳过 |
| 拿不到 usage | OpenAI 格式需加 stream_options.include_usage;Anthropic 格式在 message_start 和 message_delta 里 |