ViperEkura 3fb4b8ab13 perf: use int32 paged KV indices ...

- store page-table, request-row, and cache-location indices as int32
- preserve CUDA graph replay with bit-exact logits and KV cache coverage
- improve B=1 decode latency by 1-6% across 1K-32K contexts on L20

2026-08-15 13:17:06 +08:00

..

lib

refactor: move compiled kernel .so files into extension/lib

2026-07-31 15:36:32 +08:00

__init__.py

refactor : use factory for attention backends

2026-08-05 15:37:22 +08:00

attention_backend.py

perf: use int32 paged KV indices

2026-08-15 13:17:06 +08:00

attention_ops.py

perf: use int32 paged KV indices

2026-08-15 13:17:06 +08:00

fp8_ops.py

refactor: split fp8 into fp8_ops adapter and fp8 policy module

2026-08-14 12:25:29 +08:00

fp8.py

fix: quantize amax from raw values, not scaled fp8 values

2026-08-14 14:26:16 +08:00

loader.py

feat: add fp8 training via cublasLt dispatch

2026-08-14 00:39:49 +08:00

rotary_backend.py

refactor: unify rotary embedding interface and update docs

2026-07-31 16:52:25 +08:00

rotary_ops.py

refactor: unify rotary embedding interface and update docs

2026-07-31 16:52:25 +08:00