refactor: clean up inference design patterns

1. KVCache base: add default task_cached/task_record_hashes, remove getattr from scheduler
2. Remove page_size param from scheduler constructor (ContiguousCache-only)
3. InferenceEngine expose cache param for KVCache injection
4. Rename page_cache -> kv_cache in Executor
5. Move stream_callback from Task to TaskManager._callbacks dict
6. TaskManager.clear_queues clears callbacks
This commit is contained in:
2026-07-05 11:41:54 +08:00
parent 5416c2e8fb
commit 849e1e00a3
5 changed files with 39 additions and 50 deletions
+7
View File
@@ -303,6 +303,13 @@ class KVCache(ABC):
self, task_ids: List[str], total_len: int, device: torch.device
) -> CacheView: ...
def task_cached(self, task_id: str) -> int:
return 0
def task_record_hashes(
self, task_id: str, prompt_ids: List[int], start_logical_page: int = 0
): ...
class PageCacheView(CacheView):
"""Bundles Storage + page_table + total_len for attention layers."""