ViperEkura 50cfd0d555 perf: reduce decode overhead in scheduler and executor ...

- Precompute page_table and decode_mask on KVCache once per step in PagePool.bind_tasks, instead of per-layer in CudaBackend/TorchNativeBackend
- Skip frequency penalty history tensor construction when all penalties are 0 in Executor.execute_decode
- Omit FrequencyPenaltyStrategy from sampling pipeline when penalty is 0
- Deduplicate get_active_tasks calls in scheduler loop (3 to 1), remove redundant sorted() on decode tasks
- Benchmark (L20, bf16, CUDA backend): B=1 9.48->9.40ms (+1%), B=4 10.73->9.89ms (+8.6%), B=8 10.77->10.13ms (+6.4%)

2026-07-31 14:50:16 +08:00

..

__init__.py

feat: unify attention backend with multi-dim mask support

2026-07-30 20:38:34 +08:00

attention_backend.py

perf: reduce decode overhead in scheduler and executor

2026-07-31 14:50:16 +08:00

attention_ops.py

refactor: unify extension API to blhd layout and is_causal

2026-07-30 18:39:20 +08:00

loader.py

feat: wire up paged decode CUDA kernel to Python extension

2026-07-11 18:40:49 +08:00