docs: 重构 README 结构，全文档添加目录导航
- README 新增 Getting Started 端到端流程，整合快速开始与演示，去重精简 - 中文 README 同步英文版结构，预处理配置改用 seq 策略 - inference.md 补充 SSE 流式格式、错误响应、/stats 端点文档 - params.md 扩展为 CLI 参考，覆盖 server/generate/preprocess 参数表 - dataflow.md 拆分 tokenization/format detection/backend 子节，新增流程图 - architecture/training/inference/preprocessing 均添加目录导航 - 移除 README CI badge
This commit is contained in:
@@ -12,7 +12,6 @@
|
||||
<img src="https://img.shields.io/github/v/release/ViperEkura/AstrAI?label=Release&color=76bad9" alt="release">
|
||||
<img src="https://img.shields.io/github/stars/ViperEkura/AstrAI?style=flat&label=Stars&color=76bad9" alt="stars">
|
||||
<img src="https://img.shields.io/github/forks/ViperEkura/AstrAI?style=flat&label=Forks&color=76bad9" alt="forks">
|
||||
<img src="https://img.shields.io/github/actions/workflow/status/ViperEkura/AstrAI/tests.yml?label=CI&color=76bad9" alt="ci">
|
||||
</div>
|
||||
<br>
|
||||
|
||||
@@ -29,7 +28,8 @@
|
||||
## 📖 Table of Contents
|
||||
|
||||
- [Features](#features)
|
||||
- [Quick Start](#quick-start)
|
||||
- [Getting Started](#getting-started)
|
||||
- [Demo](#demo)
|
||||
- [Documentation](#documentation)
|
||||
- [Contributing](#contributing)
|
||||
- [Community](#community)
|
||||
@@ -50,33 +50,43 @@
|
||||
- 🤗 **HuggingFace-Style API**: AutoModel/AutoTokenizer APIs inspired by HuggingFace for easy model and tokenizer loading.
|
||||
- 🔌 **Dual API Compatibility**: Supports both OpenAI and Anthropic chat completion APIs out of the box.
|
||||
|
||||
### Quick Start
|
||||
### Getting Started
|
||||
|
||||
#### Installation
|
||||
End-to-end walkthrough in 5 steps:
|
||||
|
||||
**1. Install**
|
||||
|
||||
```bash
|
||||
git clone https://github.com/ViperEkura/AstrAI.git
|
||||
cd AstrAI
|
||||
pip install -e .
|
||||
# pip install -e ".[dev]" # optional: dev dependencies (pytest, ruff)
|
||||
```
|
||||
|
||||
For development dependencies:
|
||||
**2. Download model**
|
||||
|
||||
```bash
|
||||
pip install -e ".[dev]"
|
||||
python scripts/demo/download.py # downloads 1B checkpoint to params/
|
||||
```
|
||||
|
||||
#### Download Pre-trained Model
|
||||
**3. Preprocess data**
|
||||
|
||||
Download pre-trained model weights (1B bilingual checkpoint) to `params/`:
|
||||
Create `pretrain.json` (preprocessing config for `seq` strategy):
|
||||
|
||||
```json
|
||||
{
|
||||
"version": 1,
|
||||
"input": {"sections": [{"field": "text", "action": "train"}]},
|
||||
"preprocessing": {"max_seq_len": 2048},
|
||||
"output": {"storage_format": "bin"}
|
||||
}
|
||||
```
|
||||
|
||||
```bash
|
||||
python scripts/demo/download.py
|
||||
python scripts/tools/preprocess.py data/*.jsonl -o output/ -c pretrain.json
|
||||
```
|
||||
|
||||
Or download manually from [HuggingFace](https://huggingface.co/ViperEk/KHAOSZ) into `params/`.
|
||||
|
||||
#### Train a Model
|
||||
**4. Train**
|
||||
|
||||
```bash
|
||||
export CUDA_VISIBLE_DEVICES=0,1,2,3
|
||||
@@ -103,15 +113,54 @@ nohup python scripts/tools/train.py \
|
||||
> out.log 2> err.log &
|
||||
```
|
||||
|
||||
Full reference at [Parameter Guide](assets/docs/params.md).
|
||||
**5. Serve & query**
|
||||
|
||||
#### Generate Text
|
||||
```bash
|
||||
# Terminal 1: start server
|
||||
python scripts/tools/server.py --param_path ./params --device cuda
|
||||
|
||||
# Terminal 2: query
|
||||
curl http://localhost:8000/v1/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"messages":[{"role":"user","content":"Hello"}],"max_tokens":512}'
|
||||
```
|
||||
|
||||
### Demo
|
||||
|
||||
Check out the demos in the `scripts/demo/` folder:
|
||||
|
||||
```bash
|
||||
# Download model weights (required before running demos)
|
||||
python scripts/demo/download.py # model → params/
|
||||
|
||||
# Interactive streaming chat (multi-turn, maintains history)
|
||||
python scripts/demo/stream_chat.py
|
||||
# Type your message after >>, type !exit to quit
|
||||
|
||||
# Batch generation (5 hardcoded prompts, non-streaming)
|
||||
python scripts/demo/generate_batch.py
|
||||
|
||||
# Single-prompt autoregressive streaming
|
||||
python scripts/demo/generate_ar.py
|
||||
```
|
||||
|
||||
All generation demos use `temperature=0.8`, `top_p=0.95`, `top_k=50`, `max_tokens=2048` by default and require `params/` to contain model weights (run `download.py` first).
|
||||
|
||||
Watch a video walkthrough on [bilibili](https://www.bilibili.com/video/BV1fuLB6yEj6).
|
||||
|
||||
---
|
||||
|
||||
See [Documentation](#documentation) for full references beyond the examples above.
|
||||
|
||||
#### Text Generation
|
||||
|
||||
Batch generation from a JSONL file:
|
||||
|
||||
```bash
|
||||
python scripts/tools/generate.py \
|
||||
--param_path /path/to/model \
|
||||
--input_json_file /path/to/input.jsonl \
|
||||
--output_json_file /path/to/output.jsonl
|
||||
--param_path ./params \
|
||||
--input_json_file input.jsonl \
|
||||
--output_json_file output.jsonl
|
||||
```
|
||||
|
||||
#### Docker
|
||||
@@ -125,9 +174,6 @@ docker build -t astrai:latest .
|
||||
# Run with GPU support
|
||||
docker run --gpus all -it astrai:latest
|
||||
|
||||
# Run with specific GPUs
|
||||
docker run --gpus '"device=0,1"' -it astrai:latest
|
||||
|
||||
# Run inference server
|
||||
docker run --gpus all -p 8000:8000 astrai:latest \
|
||||
python -m scripts.tools.server --port 8000 --device cuda
|
||||
@@ -144,84 +190,37 @@ docker compose --profile cpu up -d
|
||||
|
||||
> **Note**: `--gpus all` is required for CUDA support. Without it, `torch.cuda.is_available()` will return `False`.
|
||||
|
||||
#### Start HTTP Server
|
||||
#### HTTP API Examples
|
||||
|
||||
Start the inference server with OpenAI and Anthropic-compatible HTTP API:
|
||||
Additional request examples beyond the [Getting Started](#getting-started) flow:
|
||||
|
||||
```bash
|
||||
python -m scripts.tools.server --port 8000 --device cuda
|
||||
```
|
||||
|
||||
Make requests:
|
||||
|
||||
```bash
|
||||
# OpenAI-compatible
|
||||
curl -X POST http://localhost:8000/v1/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"messages": [{"role": "user", "content": "Hello"}],
|
||||
"max_tokens": 512
|
||||
}'
|
||||
|
||||
# OpenAI-compatible streaming
|
||||
curl -X POST http://localhost:8000/v1/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"messages": [{"role": "user", "content": "Tell a story"}],
|
||||
"stream": true,
|
||||
"max_tokens": 500
|
||||
}'
|
||||
-d '{"messages":[{"role":"user","content":"Tell a story"}],"stream":true,"max_tokens":500}'
|
||||
|
||||
# Anthropic-compatible
|
||||
curl -X POST http://localhost:8000/v1/messages \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"model": "astrai",
|
||||
"system": "You are a helpful assistant.",
|
||||
"messages": [{"role": "user", "content": "Hello"}],
|
||||
"max_tokens": 512
|
||||
}'
|
||||
-d '{"model":"astrai","system":"You are a helpful assistant.","messages":[{"role":"user","content":"Hello"}],"max_tokens":512}'
|
||||
|
||||
# Anthropic-compatible streaming with stop sequences
|
||||
curl -X POST http://localhost:8000/v1/messages \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"model": "astrai",
|
||||
"messages": [{"role": "user", "content": "Write a story"}],
|
||||
"max_tokens": 500,
|
||||
"stream": true,
|
||||
"stop_sequences": ["The end"]
|
||||
}'
|
||||
-d '{"model":"astrai","messages":[{"role":"user","content":"Write a story"}],"max_tokens":500,"stream":true,"stop_sequences":["The end"]}'
|
||||
|
||||
# Health check
|
||||
curl http://localhost:8000/health
|
||||
```
|
||||
|
||||
#### Demo
|
||||
|
||||
Check out the demos in the `scripts/demo/` folder:
|
||||
|
||||
```bash
|
||||
# Download model weights (required before running demos)
|
||||
python scripts/demo/download.py
|
||||
|
||||
# Interactive streaming chat
|
||||
python scripts/demo/stream_chat.py
|
||||
|
||||
# Batch generation
|
||||
python scripts/demo/generate_batch.py
|
||||
|
||||
# Auto‑regressive generation
|
||||
python scripts/demo/generate_ar.py
|
||||
```
|
||||
|
||||
Watch a video walkthrough on [bilibili](https://www.bilibili.com/video/BV1fuLB6yEj6).
|
||||
See [Inference Guide](assets/docs/inference.md) for SSE streaming format, error codes, and stats endpoint.
|
||||
|
||||
### Documentation
|
||||
|
||||
| Document | Description |
|
||||
|----------|-------------|
|
||||
| [Parameter Guide](./assets/docs/params.md) | Training & inference parameters |
|
||||
| [CLI Reference](./assets/docs/params.md) | Parameters for all CLI tools (train, server, generate, preprocess) |
|
||||
| [Architecture](./assets/docs/architecture.md) | System architecture, class diagram & design patterns |
|
||||
| [Training](./assets/docs/training.md) | Training loop, strategies & formulas |
|
||||
| [Inference](./assets/docs/inference.md) | KVCache, continuous batching, sampling & HTTP API |
|
||||
|
||||
Reference in New Issue
Block a user