refactor: decouple task cache from PagePool and unify steady-state detection
- TaskCacheRegistry -> TaskCacheManager (independent, held by scheduler) - TaskCacheState co-locates 5 parallel dicts into one dataclass - AllocationStrategy base class + PagedStrategy subclass (page_size is a parameter) - _rollback() helper for unified cleanup (no duplicate free paths) - Task._kv_len + prefill_done property (explicit, no output_tokens proxy) - Steady-state detection single-sourced in TaskCacheManager.bind() - PagePool is now pure physical layer (no task knowledge) - Removed dead _page_to_hash dict in RadixCache
This commit is contained in:
@@ -20,11 +20,12 @@ def test_task_default_status_is_pending():
|
||||
def test_task_next_pos():
|
||||
task = Task("id1", [1, 2, 3])
|
||||
task.input_tokens = 5
|
||||
task.mark_prefill_done()
|
||||
assert task.next_pos == 5
|
||||
task.output_ids.append(4)
|
||||
assert task.next_pos == 5
|
||||
task.output_ids.append(5)
|
||||
task.advance_kv()
|
||||
assert task.next_pos == 6
|
||||
task.advance_kv()
|
||||
assert task.next_pos == 7
|
||||
|
||||
|
||||
def test_task_is_finished_max_tokens():
|
||||
|
||||
Reference in New Issue
Block a user