ViperEkura ae9fd546ef perf: merge prefill warmup into _warmup_cuda_graphs ...

- 64-token prefill forward triggers cuBLAS auto-tuning at init
- reduces first-chat prefill from ~520ms to ~27ms
- warmup decode also drops from ~215ms to ~71ms

2026-08-08 13:10:58 +08:00

..

__init__.py

feat: add per-task throughput and latency metrics

2026-08-08 12:10:06 +08:00

cache.py

refactor: extract shared steady-state increment detection

2026-08-07 23:00:25 +08:00

executor.py

perf: merge prefill warmup into _warmup_cuda_graphs

2026-08-08 13:10:58 +08:00

graph.py

fix: add out_buf to attn_paged_decode for CUDA graph capture compatibility

2026-08-07 19:45:59 +08:00

metrics.py

feat: add per-task throughput and latency metrics

2026-08-08 12:10:06 +08:00

scheduler.py

feat: add per-task throughput and latency metrics

2026-08-08 12:10:06 +08:00

task.py

feat: add per-task throughput and latency metrics

2026-08-08 12:10:06 +08:00

workspace.py

fix: add out_buf to attn_paged_decode for CUDA graph capture compatibility

2026-08-07 19:45:59 +08:00