ViperEkura d0e5d910de perf: reduce remaining per-step allocations ...

- hoist prefill qo_indptr into the workspace so CudaBackend.fwd_prefill does not rebuild it per layer
- cache has_freq in SamplingBatchInfo to drop the per-step GPU any() sync
- drop pin_memory host staging for input_ids; sync copy suffices for a small batch

2026-08-03 01:10:06 +08:00

..

lib

refactor: move compiled kernel .so files into extension/lib

2026-07-31 15:36:32 +08:00

__init__.py

refactor: replace magic layout ints with TensorLayout enum

2026-08-01 11:05:52 +08:00

attention_backend.py

perf: reduce remaining per-step allocations

2026-08-03 01:10:06 +08:00

attention_ops.py

feat: SGLang-style paged attention kernels replace page-table path

2026-08-01 15:41:25 +08:00

loader.py

feat: SGLang-style paged attention kernels replace page-table path

2026-08-01 15:41:25 +08:00

rotary_backend.py

refactor: unify rotary embedding interface and update docs

2026-07-31 16:52:25 +08:00

rotary_ops.py

refactor: unify rotary embedding interface and update docs

2026-07-31 16:52:25 +08:00