feat: unify Docker serving configuration in YAML

- Add server.py --config serve.yaml; explicit CLI flags override YAML
- Add scripts/serve.sh and serve_runtime.py for the Compose lifecycle
- Template server/cpu ports and param mounts in docker-compose.yml
- Document schema in docs/developer/docker-serving.md and params guide
- Add tests for runtime parsing and server CLI merge logic
This commit is contained in:
2026-08-21 23:16:45 +08:00
parent dcc96de12a
commit cb21af38ba
10 changed files with 817 additions and 6 deletions
+17
View File
@@ -203,6 +203,7 @@ nohup python scripts/tools/train.py \
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `--config`, `-c` | path | `None` | Serving YAML config. CLI flags override YAML values |
| `--host` | str | `0.0.0.0` | Host address |
| `--port` | int | `8000` | Port number |
| `--param_path` | path | `project_root/params` | Path to model parameters |
@@ -217,6 +218,22 @@ Usage:
python scripts/tools/server.py --param_path ./params --device cuda --dtype bfloat16
```
YAML config (a `server:` section; explicit CLI flags override YAML values):
```bash
python scripts/tools/server.py --config serve.yaml
```
```yaml
server:
host: 0.0.0.0
port: 8000
device: cuda
dtype: bfloat16
max_batch_size: 16
max_seq_len: null
```
`serve.yaml` also carries a `runtime:` section for the Docker wrapper; see
[Docker Serving](../developer/docker-serving.md).
See [Inference Guide](inference.md) for HTTP API documentation.
## Generate (`generate.py`)