4 Commits

Include renames

Author	SHA1	Message	Date
ViperEkura	2bc4d2b8a8	refactor: unify kernel module loading and packaging ... - loader.py: lazy/cached import; is_available defers the actual load; get_module raises on unavailable - ops/{attention,rotary,fp8}: use get_module instead of touching private _modules or their own _mod() cache - package-data: ship astrai.extension.lib *.so in built wheels (non-editable installs previously lost every kernel)	2026-08-23 15:57:04 +08:00
ViperEkura	3d3ea47d37	refactor: standardize packed 3d inference ... - keep training attention on dense 4d tensors - use packed 3d tensors with KV cache for inference - extend CUDA rotary embedding to packed 3d inputs - adapt torch, CUDA and FlashAttention backend dispatch Benchmark: NVIDIA L20, BF16, 1B model, paged KV cache, CUDA Graph, prompt 512, generation 128 (median of 3 alternating runs) - batch 1: 234.5 -> 242.6 tok/s (1.034x, +3.4%) - batch 8: 1243.1 -> 1286.6 tok/s (1.035x, +3.5%)	2026-08-19 00:36:53 +08:00
ViperEkura	1bcd8f53ab	perf: precompute ragged Q tile scheduling	2026-08-16 23:32:46 +08:00
ViperEkura	6ac3b51496	refactor: separate extension ops and backends	2026-08-16 21:15:52 +08:00