docs: fix stale developer documentation claims

- Move task_alloc/task_free/task_extend/task_cached/task_record_hashes and bind from the PagePool card to a new TaskCacheManager card matching pool.py
- Drop the nonexistent Executor tokenizer attribute and association, add task_cache instead
- Add AllocationStrategy/ContiguousStrategy/PagedStrategy cards and point Allocator/RadixCache composition at PagedStrategy
- Add TaskCacheManager and the allocation strategies to the module overview, add _task_cache to InferenceScheduler
- Fix the design-pattern count in the table of contents (15 -> 16)
- Rewrite the FlashAttnBackend class docstring: packed decode gathers flat K/V via req_to_token and calls flash_attn_varlen_func; dense prefill uses flash_attn_func (no flash_attn_with_kvcache exists)
- Apply the same correction to the backend bullets in internals.md and cuda_kernels.md
- Rename the stale fp8_mma_test.cu reference to fp8_test.cu in cuda_kernels.md
This commit is contained in:
2026-08-31 14:24:51 +08:00
parent a7d4cb25c5
commit e3c3e28a11
4 changed files with 51 additions and 19 deletions
+6 -5
View File
@@ -694,12 +694,13 @@ class CudaBackend(AttentionBackend):
class FlashAttnBackend(AttentionBackend):
"""FlashAttention backend via the optional ``flash-attn`` package.
Decode (q_len=1, contiguous cache): uses ``flash_attn_with_kvcache``,
which reads K/V directly from the flat pool via cache_batch_idx +
cache_seqlens — no materialized KV gather.
Decode (q_len=1, contiguous cache): writes K/V to the pool, gathers
flat K/V via the ``req_to_token`` page table, and calls
``flash_attn_varlen_func`` over the ragged batch
(``qo_indptr``/``kv_indptr``).
Prefill / non-contiguous decode: falls back to KV gather +
``flash_attn_func``.
Prefill: packed 3-D calls share the ``flash_attn_varlen_func`` path;
dense 4-D calls go through ``flash_attn_func`` (mask-free only).
"""
@classmethod