refactor: decouple task cache from PagePool and unify steady-state detection

- TaskCacheRegistry -> TaskCacheManager (independent, held by scheduler)
- TaskCacheState co-locates 5 parallel dicts into one dataclass
- AllocationStrategy base class + PagedStrategy subclass (page_size is a parameter)
- _rollback() helper for unified cleanup (no duplicate free paths)
- Task._kv_len + prefill_done property (explicit, no output_tokens proxy)
- Steady-state detection single-sourced in TaskCacheManager.bind()
- PagePool is now pure physical layer (no task knowledge)
- Removed dead _page_to_hash dict in RadixCache
This commit is contained in:
2026-08-08 22:43:06 +08:00
parent ca50fe4721
commit 3fa7e66676
10 changed files with 474 additions and 400 deletions
+2
View File
@@ -7,6 +7,7 @@ from astrai.inference.core.cache import (
PagePool,
RadixCache,
ReqToTokenPool,
TaskCacheManager,
page_hash,
)
from astrai.inference.core.executor import Executor
@@ -21,6 +22,7 @@ __all__ = [
"PagePool",
"RadixCache",
"ReqToTokenPool",
"TaskCacheManager",
"page_hash",
"Executor",
"InferenceScheduler",