AstrAI

Commit Graph

Author	SHA1	Message	Date
ViperEkura	d08a92c7bd	feat: add frequency penalty to inference sampling pipeline - Add FrequencyPenaltyStrategy (logit -= penalty * count) - Per-task rep_window for penalty history lookup - Wire through engine, task, executor, API layer - Add --frequency_penalty and --rep_window to stream_chat.py - 9 unit tests for frequency penalty strategy	2026-07-17 21:28:31 +08:00
ViperEkura	e3382f6bb5	fix: 修复推理引擎 batch decode 中多项正确性与并发问题 - scheduler: decode 分组由幂次分桶改为精确 next_pos，消除 KV cache 位置错乱 - task: activate() 加锁操作 active_tasks，消除数据竞争 - engine: wait_completion 加超时，防止分配失败时永久死锁 - sample: TopKStrategy 向量化为 per-sample threshold，尊重各 task 的 top_k - cache: Storage.write/gather 中 -1 页改用 mask 处理，防数据污染 - executor: prefill 逐 task 循环改为单次 tensor 调用	2026-05-14 21:31:39 +08:00
ViperEkura	7d4029c2a4	test: inference 模块补全单元测试，cache/sample/engine/task - test_cache: page_hash, PagePool, PrefixCache, TaskTable, PagedCache write/gather - test_sample: TemperatureStrategy, TopKStrategy, TopPStrategy, SamplingPipeline, sample() - test_engine: _Result 线程安全, generate stream/non-stream batch/single - test_task: Task 生命周期, TaskManager 队列操作 - 4 新文件, +771 行, 116 total tests	2026-05-12 12:17:57 +08:00

Author

SHA1

Message

Date

ViperEkura

d08a92c7bd

feat: add frequency penalty to inference sampling pipeline

- Add FrequencyPenaltyStrategy (logit -= penalty * count)
- Per-task rep_window for penalty history lookup
- Wire through engine, task, executor, API layer
- Add --frequency_penalty and --rep_window to stream_chat.py
- 9 unit tests for frequency penalty strategy

2026-07-17 21:28:31 +08:00

ViperEkura

e3382f6bb5

fix: 修复推理引擎 batch decode 中多项正确性与并发问题

- scheduler: decode 分组由幂次分桶改为精确 next_pos，消除 KV cache 位置错乱
- task: activate() 加锁操作 active_tasks，消除数据竞争
- engine: wait_completion 加超时，防止分配失败时永久死锁
- sample: TopKStrategy 向量化为 per-sample threshold，尊重各 task 的 top_k
- cache: Storage.write/gather 中 -1 页改用 mask 处理，防数据污染
- executor: prefill 逐 task 循环改为单次 tensor 调用

2026-05-14 21:31:39 +08:00

ViperEkura

7d4029c2a4

test: inference 模块补全单元测试，cache/sample/engine/task

- test_cache: page_hash, PagePool, PrefixCache, TaskTable, PagedCache write/gather
- test_sample: TemperatureStrategy, TopKStrategy, TopPStrategy, SamplingPipeline, sample()
- test_engine: _Result 线程安全, generate stream/non-stream batch/single
- test_task: Task 生命周期, TaskManager 队列操作
- 4 新文件, +771 行, 116 total tests

2026-05-12 12:17:57 +08:00

3 Commits