commit 110226d612b809d9c7f518411be85e60ad9fe0fd Author: jiachun <2609024259@qq.com> Date: Sun Jun 14 18:54:46 2026 +0800 first commit diff --git a/README.md b/README.md new file mode 100644 index 0000000..52573ed --- /dev/null +++ b/README.md @@ -0,0 +1,45 @@ +# GLM-4.6V-Flash Server + +基于 vLLM 的 OpenAI 兼容 API 服务。 + +## Requirements + +- NVIDIA GPU with ≥44 GiB memory (e.g. L20) +- CUDA driver ≥570.x (CUDA 12.8) +- Python 3.12 + +## Installation + +```bash +bash install.sh +``` + +或手动三步: + +```bash +pip install torch==2.10.0 torchvision==0.25.0 \ + --index-url https://download.pytorch.org/whl/cu128 +pip install vllm==0.18.0 flashinfer-python==0.6.6 +pip install transformers==5.12.0 +``` + +## 启动 + +```bash +python server.py +``` + +或指定 GPU: + +```bash +CUDA_VISIBLE_DEVICES=2 python server.py +``` + +## 验证 + +```bash +curl http://localhost:8000/v1/models +curl http://localhost:8000/v1/chat/completions \ + -H "Content-Type: application/json" \ + -d '{"model":"GLM-4.6V-Flash","messages":[{"role":"user","content":"hello"}]}' +```