AstrAI

Files

T

ViperEkura ca50fe4721 refactor: remove inference redundancy and fix cache leaks

- drop Executor unused tokenizer field, _head_dim, stale metrics docstring
- unify greedy sampling via SamplingPipeline.sample, drop top-level duplicate
- drop Task.flush_remaining no-op and unreachable prompt-length branch
- drop ProtocolHandler redundant chunks list (reuse body)
- fix page_size=1 token-slot leak on task_free
- clear _task_pages/_task_slots on alloc-failure paths
- reset _bind_state on task_free to avoid stale steady-state reuse
- remove unreachable contiguous branches in paged-only helpers

2026-08-08 21:45:01 +08:00

__init__.py

feat: add per-task throughput and latency metrics

2026-08-08 12:10:06 +08:00

cache.py

refactor: remove inference redundancy and fix cache leaks

2026-08-08 21:45:01 +08:00

executor.py

refactor: remove inference redundancy and fix cache leaks

2026-08-08 21:45:01 +08:00

graph.py

fix: add out_buf to attn_paged_decode for CUDA graph capture compatibility