feat: 新增 Anthropic 兼容 /v1/messages API,移除旧版 /generate 端点

- 新增 /v1/messages 端点,兼容 Anthropic Messages API 格式
- 支持流式 SSE(message_start → content_block_delta → message_stop)
- 支持 system 顶层提示词与 stop_sequences 停止序列
- 新增 AnthropicMessage / MessagesRequest Pydantic 模型
- 移除旧版 /generate 端点及相关测试用例
- 更新 README.md / README-zh-CN.md / introduction.md 文档
This commit is contained in:
2026-05-09 11:47:22 +08:00
parent 9d96b0431d
commit d73f52a2f8
5 changed files with 377 additions and 153 deletions
+32 -22
View File
@@ -47,6 +47,7 @@
- 📦 **Lightweight**: Minimal dependencies, easy to deploy.
- 🔬 **ResearchFriendly**: Modular design, easy to experiment with new ideas.
- 🤗 **HuggingFace Integration**: Compatible with HuggingFace models and datasets.
- 🔌 **Dual API Compatibility**: Supports both OpenAI and Anthropic chat completion APIs out of the box.
### Quick Start
@@ -67,27 +68,9 @@ pip install -e ".[dev]"
#### Train a Model
```bash
python scripts/tools/train.py \
--train_type=seq \
--data_root_path=/path/to/dataset \
--param_path=/path/to/model \
--n_epoch=3 \
--batch_size=4 \
--accumulation_steps=8 \
--max_lr=3e-4 \
--warmup_steps=2000 \
--ckpt_interval=5000 \
--ckpt_dir=./checkpoints
python scripts/tools/train.py --train_type=seq --data_root_path=/path/to/dataset --param_path=/path/to/model
```
#### Generate Text
```bash
python scripts/tools/generate.py --param_path=/path/to/param_path
```
#### Training Parameters
| Parameter | Description | Default |
|-----------|-------------|---------|
| `--train_type` | Training type (`seq`, `sft`, `dpo`) | required |
@@ -105,6 +88,12 @@ python scripts/tools/generate.py --param_path=/path/to/param_path
Full reference at [Parameter Guide](./assets/docs/params.md#training-parameters).
#### Generate Text
```bash
python scripts/tools/generate.py --param_path=/path/to/param_path
```
#### Docker
Build and run with Docker (recommended for GPU environments):
@@ -131,7 +120,7 @@ docker run --gpus all -v /path/to/data:/data -it astrai:latest
#### Start HTTP Server
Start the inference server with OpenAI-compatible HTTP API:
Start the inference server with OpenAI and Anthropic-compatible HTTP API:
```bash
python -m scripts.tools.server --port 8000 --device cuda
@@ -140,7 +129,7 @@ python -m scripts.tools.server --port 8000 --device cuda
Make requests:
```bash
# Chat API (OpenAI compatible)
# OpenAI-compatible
curl -X POST http://localhost:8000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
@@ -148,7 +137,7 @@ curl -X POST http://localhost:8000/v1/chat/completions \
"max_tokens": 512
}'
# Streaming response
# OpenAI-compatible streaming
curl -X POST http://localhost:8000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
@@ -157,6 +146,27 @@ curl -X POST http://localhost:8000/v1/chat/completions \
"max_tokens": 500
}'
# Anthropic-compatible
curl -X POST http://localhost:8000/v1/messages \
-H "Content-Type: application/json" \
-d '{
"model": "astrai",
"system": "You are a helpful assistant.",
"messages": [{"role": "user", "content": "Hello"}],
"max_tokens": 512
}'
# Anthropic-compatible streaming with stop sequences
curl -X POST http://localhost:8000/v1/messages \
-H "Content-Type: application/json" \
-d '{
"model": "astrai",
"messages": [{"role": "user", "content": "Write a story"}],
"max_tokens": 500,
"stream": true,
"stop_sequences": ["The end"]
}'
# Health check
curl http://localhost:8000/health
```