refactor: decouple task cache from PagePool and unify steady-state detection

- TaskCacheRegistry -> TaskCacheManager (independent, held by scheduler)
- TaskCacheState co-locates 5 parallel dicts into one dataclass
- AllocationStrategy base class + PagedStrategy subclass (page_size is a parameter)
- _rollback() helper for unified cleanup (no duplicate free paths)
- Task._kv_len + prefill_done property (explicit, no output_tokens proxy)
- Steady-state detection single-sourced in TaskCacheManager.bind()
- PagePool is now pure physical layer (no task knowledge)
- Removed dead _page_to_hash dict in RadixCache
This commit is contained in:
2026-08-08 22:43:06 +08:00
parent ca50fe4721
commit 3fa7e66676
10 changed files with 474 additions and 400 deletions
+2
View File
@@ -38,6 +38,7 @@ from astrai.inference.core import (
RadixCache,
ReqToTokenPool,
Task,
TaskCacheManager,
TaskManager,
TaskStatus,
page_hash,
@@ -67,6 +68,7 @@ __all__ = [
"PagePool",
"RadixCache",
"ReqToTokenPool",
"TaskCacheManager",
"page_hash",
"sample",
"BaseSamplingStrategy",