4 Commits

Include renames

Author	SHA1	Message	Date
ViperEkura	47b3ed4e44	feat: propagate attention backend across scheduler threads ... - InferenceEngine/Scheduler accept an explicit backend - capture request-level attn_backend context onto Task - split prefill/decode batches by backend instance - ASTR_BACKEND env overrides ContextVar as process-wide policy - report resolved backend and CUDA-graph state in benchmark	2026-08-09 13:32:40 +08:00
ViperEkura	c1d05ae11d	perf: benchmark decode via real inference engine ... - route decode benchmark through InferenceEngine generate path - add enable_cuda_graph toggle to engine, scheduler, and executor - make benchmark --cuda-graph/--no-cuda-graph control the toggle - hoist local time imports to module top	2026-08-09 11:47:14 +08:00
ViperEkura	be90dfe2bd	fix: isolate continuous batch decode state ... - Match steady-state metadata to the active task IDs - Rebuild request mappings for cached prefix pages - Add regressions for batch refill and prefix reuse	2026-08-09 01:01:41 +08:00
ViperEkura	0c1b7664c1	refactor: split infer core into subpackages by concern ... - Eliminate core/ directory into cache/, runtime/, network/ subpackages plus flat modules - Split cache.py (647 lines) into cache/{buffer,strategy,pool}.py by layer - Add explicit ContiguousStrategy, make AllocationStrategy a real ABC - Move TaskCacheState to cache/strategy.py, drop string forward references - Rename api/ to network/, server.py to app.py - Move sample.py into runtime/ alongside executor and graph - Simplify TaskCacheManager.__init__ to single pool param - Expose pool.strategy and pool.req_pool as public properties - Fix KVCache import in attention_backend.py (TYPE_CHECKING guard) - Fix steady-state decode reading uninitialized position_ids on first step	2026-08-08 23:43:05 +08:00