ViperEkura
d08a92c7bd
feat: add frequency penalty to inference sampling pipeline
...
- Add FrequencyPenaltyStrategy (logit -= penalty * count)
- Per-task rep_window for penalty history lookup
- Wire through engine, task, executor, API layer
- Add --frequency_penalty and --rep_window to stream_chat.py
- 9 unit tests for frequency penalty strategy
2026-07-17 21:28:31 +08:00
ViperEkura
a5c1de6b1b
feat: add model_path temperature top_p top_k max_tokens system_prompt args to stream_chat
2026-07-04 07:33:32 +08:00
ViperEkura
ad9f4d9cf6
refactor: generate_ar 改用流式输出并去除冗余注释
2026-05-17 10:23:42 +08:00
ViperEkura
f0339022c1
fix: batch 推理示例添加 chat template 和 system prompt
...
- 新增 prompts 列表，对每个输入应用 apply_chat_template
- 添加 system message 到对话模板
2026-05-14 20:59:01 +08:00
ViperEkura
44d7a4e959
refactor: 设计模式优化 inference 模块导入结构
...
- 新建 cache.py：SlotAllocator 对象池 + PrefixCacheManager
- 新建 sampling.py：Temperature/TopK/TopP 可组合策略
- TaskStatus 改用 Enum，GenerationParams 值对象模式
- _STOP 移至 cache.py，解除 engine→scheduler 轻量耦合
- 更新测试导入路径，ruff 格式检查通过
2026-05-08 16:57:57 +08:00
ViperEkura
cb0e7f2a80
build: 修改docker 构建流程
2026-04-10 11:25:00 +08:00
ViperEkura
39766aa1dc
chore: 修改类名，优化导入顺序
2026-04-05 22:27:57 +08:00
ViperEkura
9b22b1651e
refactor: 优化工具脚本接口并修复批处理问题
2026-04-05 21:56:22 +08:00
ViperEkura
fc278d17ab
feat: 实现模型动态注册机制
2026-04-05 19:38:12 +08:00
ViperEkura
861d33b1a1
refactor: 更新inference 部分的实现
2026-04-04 23:49:18 +08:00
ViperEkura
0852b852f8
refactor: 优化参数传递，清理导入样式
2026-04-03 22:06:32 +08:00
ViperEkura
912d7c7f54
chore: 更新脚本并且修改gitignore
2026-04-02 15:40:31 +08:00
ViperEkura
9f1561afe7
reafactor: 修改ModelParameter
2026-03-31 16:00:55 +08:00
ViperEkura
4ead0a20cf
chore: 修改文件夹结构
2026-03-31 10:14:08 +08:00
ViperEkura