ToolAgent

基于 vLLM + opencode 的 AI 编程助手服务。

Requirements

• NVIDIA GPU with ≥24 GiB memory (e.g. L20)
• CUDA driver ≥570.x (CUDA 12.8)
• Python 3.12

Installation

bash install.sh

或手动三步：

pip install torch==2.10.0 torchvision==0.25.0 \
  --index-url https://download.pytorch.org/whl/cu128
pip install vllm==0.18.0 flashinfer-python==0.6.6
pip install transformers==5.12.0

启动

一键启动

./start.sh          # 前后端同时启动
./start.sh backend  # 仅后端
./start.sh frontend # 仅前端
./start.sh all 3    # 指定 GPU 3

手动启动

后端 (vLLM)

source /home/kxqandccx/miniconda3/bin/activate agent_use
CUDA_VISIBLE_DEVICES=2 nohup python server.py > server.log 2>&1 &

模型加载约 30-60 秒。日志在 server.log。

前端 (opencode web)

cd /home/kxqandccx/kxq/llm_tool_use
nohup opencode web --port 3000 > web.log 2>&1 &

访问 http://127.0.0.1:3000/。

验证

# 查看可用模型
curl http://localhost:8000/v1/models

# 文本对话
curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"Qwen2.5-7B-Instruct","messages":[{"role":"user","content":"你好"}]}'

# 工具调用
curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model":"Qwen2.5-7B-Instruct",
    "messages":[{"role":"user","content":"北京天气"}],
    "tools":[{"type":"function","function":{"name":"get_weather","description":"获取天气","parameters":{"type":"object","properties":{"city":{"type":"string"}},"required":["city"]}}}],
    "tool_choice":"auto"
  }'