ViperEkura d0e5d910de perf: reduce remaining per-step allocations ...

- hoist prefill qo_indptr into the workspace so CudaBackend.fwd_prefill does not rebuild it per layer
- cache has_freq in SamplingBatchInfo to drop the per-step GPU any() sync
- drop pin_memory host staging for input_ids; sync copy suffices for a small batch

2026-08-03 01:10:06 +08:00

..

api

refactor: migrate config system to Pydantic dataclasses

2026-07-30 08:25:32 +08:00

core

perf: reduce remaining per-step allocations

2026-08-03 01:10:06 +08:00

__init__.py

refactor: rebuild KV cache with three-layer separation architecture

2026-07-30 17:19:06 +08:00

engine.py

refactor: rebuild KV cache with three-layer separation architecture

2026-07-30 17:19:06 +08:00

sample.py

perf: preallocate inference decode buffers

2026-08-03 00:55:26 +08:00