46 lines
788 B
Markdown
46 lines
788 B
Markdown
# GLM-4.6V-Flash Server
|
||
|
||
基于 vLLM 的 OpenAI 兼容 API 服务。
|
||
|
||
## Requirements
|
||
|
||
- NVIDIA GPU with ≥44 GiB memory (e.g. L20)
|
||
- CUDA driver ≥570.x (CUDA 12.8)
|
||
- Python 3.12
|
||
|
||
## Installation
|
||
|
||
```bash
|
||
bash install.sh
|
||
```
|
||
|
||
或手动三步：
|
||
|
||
```bash
|
||
pip install torch==2.10.0 torchvision==0.25.0 \
|
||
--index-url https://download.pytorch.org/whl/cu128
|
||
pip install vllm==0.18.0 flashinfer-python==0.6.6
|
||
pip install transformers==5.12.0
|
||
```
|
||
|
||
## 启动
|
||
|
||
```bash
|
||
python server.py
|
||
```
|
||
|
||
或指定 GPU：
|
||
|
||
```bash
|
||
CUDA_VISIBLE_DEVICES=2 python server.py
|
||
```
|
||
|
||
## 验证
|
||
|
||
```bash
|
||
curl http://localhost:8000/v1/models
|
||
curl http://localhost:8000/v1/chat/completions \
|
||
-H "Content-Type: application/json" \
|
||
-d '{"model":"GLM-4.6V-Flash","messages":[{"role":"user","content":"hello"}]}'
|
||
```
|