perf: dispatch linear gemv by decode batch size and unify extension style
- replace the per-shape auto tables in the linear backend with an M-banded rule (M in [2,4] on compute capability 8.0+) that measured at the HBM bandwidth floor across every family, and fold the capability check into the capable guard - drop the unreachable swiglu auto shape-table machinery so both backends share one env-mode ladder via the new dispatch.env_mode helper - add __all__ across extension modules, name the rotary registration records, and unify typing to the typing-module style - rewrite test_linear_dispatch.py around behavioral routing assertions and document the M-banded policy in the developer docs - Benchmark: L20 SM89, Python dispatch overhead 2.9us to 1.5us, auto now covers every projection shape at M in [2,4].
This commit is contained in:
@@ -34,8 +34,9 @@ The kernel suite compares the directly callable primitive with `F.linear`.
|
||||
Use repeatable `--shape-label` and `--chain-label` filters for a focused run.
|
||||
The synthetic-chain suite alternates `ASTRAI_GEMV=0` and `auto`, includes
|
||||
dependent MLP work and Python dispatch, and rotates through distinct weights.
|
||||
Pass `--candidate-mode 1` to characterize a family before adding it to the
|
||||
automatic shape table; the checked-in final evidence always uses `auto`.
|
||||
Automatic dispatch is keyed on the decode batch size alone (`M` in `[2, 4]` on
|
||||
compute capability 8.0+); use `--candidate-mode 1` to characterize a family
|
||||
before widening that band. The checked-in final evidence always uses `auto`.
|
||||
It is deliberately not labeled a whole-model throughput benchmark. Both
|
||||
suites report median/p90 CUDA-event latency plus maximum absolute error,
|
||||
relative L2 error, and row-wise argmax parity.
|
||||
|
||||
Reference in New Issue
Block a user