update README with Qwen2.5 and manual start
This commit is contained in:
parent
fe997f6522
commit
b4490f26e2
52
README.md
52
README.md
|
|
@ -1,10 +1,10 @@
|
||||||
# GLM-4.6V-Flash Server
|
# ToolAgent
|
||||||
|
|
||||||
基于 vLLM 的 OpenAI 兼容 API 服务。
|
基于 vLLM + opencode 的 AI 编程助手服务。
|
||||||
|
|
||||||
## Requirements
|
## Requirements
|
||||||
|
|
||||||
- NVIDIA GPU with ≥44 GiB memory (e.g. L20)
|
- NVIDIA GPU with ≥24 GiB memory (e.g. L20)
|
||||||
- CUDA driver ≥570.x (CUDA 12.8)
|
- CUDA driver ≥570.x (CUDA 12.8)
|
||||||
- Python 3.12
|
- Python 3.12
|
||||||
|
|
||||||
|
|
@ -25,21 +25,53 @@ pip install transformers==5.12.0
|
||||||
|
|
||||||
## 启动
|
## 启动
|
||||||
|
|
||||||
```bash
|
### 一键启动
|
||||||
python server.py
|
|
||||||
```
|
|
||||||
|
|
||||||
或指定 GPU:
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
CUDA_VISIBLE_DEVICES=2 python server.py
|
./start.sh # 前后端同时启动
|
||||||
|
./start.sh backend # 仅后端
|
||||||
|
./start.sh frontend # 仅前端
|
||||||
|
./start.sh all 3 # 指定 GPU 3
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### 手动启动
|
||||||
|
|
||||||
|
#### 后端 (vLLM)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
source /home/kxqandccx/miniconda3/bin/activate agent_use
|
||||||
|
CUDA_VISIBLE_DEVICES=2 nohup python server.py > server.log 2>&1 &
|
||||||
|
```
|
||||||
|
|
||||||
|
模型加载约 30-60 秒。日志在 `server.log`。
|
||||||
|
|
||||||
|
#### 前端 (opencode web)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd /home/kxqandccx/kxq/llm_tool_use
|
||||||
|
nohup opencode web --port 3000 > web.log 2>&1 &
|
||||||
|
```
|
||||||
|
|
||||||
|
访问 `http://127.0.0.1:3000/`。
|
||||||
|
|
||||||
## 验证
|
## 验证
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
# 查看可用模型
|
||||||
curl http://localhost:8000/v1/models
|
curl http://localhost:8000/v1/models
|
||||||
|
|
||||||
|
# 文本对话
|
||||||
curl http://localhost:8000/v1/chat/completions \
|
curl http://localhost:8000/v1/chat/completions \
|
||||||
-H "Content-Type: application/json" \
|
-H "Content-Type: application/json" \
|
||||||
-d '{"model":"GLM-4.6V-Flash","messages":[{"role":"user","content":"hello"}]}'
|
-d '{"model":"Qwen2.5-7B-Instruct","messages":[{"role":"user","content":"你好"}]}'
|
||||||
|
|
||||||
|
# 工具调用
|
||||||
|
curl http://localhost:8000/v1/chat/completions \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-d '{
|
||||||
|
"model":"Qwen2.5-7B-Instruct",
|
||||||
|
"messages":[{"role":"user","content":"北京天气"}],
|
||||||
|
"tools":[{"type":"function","function":{"name":"get_weather","description":"获取天气","parameters":{"type":"object","properties":{"city":{"type":"string"}},"required":["city"]}}}],
|
||||||
|
"tool_choice":"auto"
|
||||||
|
}'
|
||||||
```
|
```
|
||||||
|
|
|
||||||
Loading…
Reference in New Issue