AstrAI/astrai
ViperEkura 5416c2e8fb perf: replace paged KV cache with contiguous ContiguousCache, decode all groups
- Add KVCache/CacheView abstract base classes in cache.py
- Add ContiguousCache (contiguous per-slot buffer, default) alongside PageCache (paged, renamed from old KVCache)
- Merge make_table_tensor + bind into bind_tasks on KVCache interface
- Remove task_cached/task_record_hashes from base class (PageCache-only)
- Scheduler: decode all position groups instead of just the largest (eliminates 63% group skip rate)
- Scheduler: accept optional cache param for swapping implementations
- Model layer type hints use CacheView base class
- Batch 1-32: 1-7% speedup from eliminating Storage.gather overhead
- All 183 inference tests pass
2026-07-05 11:34:36 +08:00
..
config fix : SFT pipeline position_ids default & doc boundary preservation 2026-07-04 15:59:11 +08:00
dataset feat: add JSONL dataset store with on-the-fly tokenization 2026-07-04 15:42:33 +08:00
inference perf: replace paged KV cache with contiguous ContiguousCache, decode all groups 2026-07-05 11:34:36 +08:00
model perf: replace paged KV cache with contiguous ContiguousCache, decode all groups 2026-07-05 11:34:36 +08:00
parallel refactor : grad_norm 指标简化,clip_grad_norm 移至 executor 2026-06-30 14:59:43 +08:00
preprocessing fix : SFT pipeline position_ids default & doc boundary preservation 2026-07-04 15:59:11 +08:00
serialization feat: add JSONL dataset store with on-the-fly tokenization 2026-07-04 15:42:33 +08:00
tokenize refactor : Pipeline 去除去重,ids 重命名为 sequence,泛型透传 2026-05-31 15:14:27 +08:00
trainer fix: scheduler division-by-zero, loss_mask bool 2026-07-03 22:04:55 +08:00
__init__.py chore : 升级 torch 2.11.0+cu128,移除自定义 Muon,修复 gloo device_id 2026-06-27 16:10:37 +08:00
factory.py fix: ForwardRef._evaluate Python 3.12 compatibility 2026-07-03 18:41:19 +08:00
protocols.py refactor: 重构训练后端为 Executor 模式 2026-05-24 20:35:44 +08:00