From 28d11f1610c5ed984ad2623f2f8052eee9184c05 Mon Sep 17 00:00:00 2001 From: ViperEkura <3081035982@qq.com> Date: Thu, 3 Sep 2026 21:06:33 +0800 Subject: [PATCH] chore: relocate kernel benchmarks to csrc/bench - Move benchmark_gemv.py, benchmark_swiglu.py, and benchmark_gemv_common.py from scripts/tools/ to csrc/bench/ so kernel benchmarks live next to the kernels they measure - Update reproduction commands in decode_linear_benchmark.md, swiglu_benchmark.md, and cuda_kernels.md - Codify the placement convention in AGENTS.md: kernel benchmarks in csrc/bench/, pure-CUDA harnesses in csrc/tests/*.cu, engine and evaluation benchmarks in scripts/ --- {scripts/tools => csrc/bench}/benchmark_gemv.py | 0 {scripts/tools => csrc/bench}/benchmark_gemv_common.py | 0 {scripts/tools => csrc/bench}/benchmark_swiglu.py | 0 docs/developer/cuda_kernels.md | 2 +- docs/developer/decode_linear_benchmark.md | 6 +++--- docs/developer/swiglu_benchmark.md | 4 ++-- 6 files changed, 6 insertions(+), 6 deletions(-) rename {scripts/tools => csrc/bench}/benchmark_gemv.py (100%) rename {scripts/tools => csrc/bench}/benchmark_gemv_common.py (100%) rename {scripts/tools => csrc/bench}/benchmark_swiglu.py (100%) diff --git a/scripts/tools/benchmark_gemv.py b/csrc/bench/benchmark_gemv.py similarity index 100% rename from scripts/tools/benchmark_gemv.py rename to csrc/bench/benchmark_gemv.py diff --git a/scripts/tools/benchmark_gemv_common.py b/csrc/bench/benchmark_gemv_common.py similarity index 100% rename from scripts/tools/benchmark_gemv_common.py rename to csrc/bench/benchmark_gemv_common.py diff --git a/scripts/tools/benchmark_swiglu.py b/csrc/bench/benchmark_swiglu.py similarity index 100% rename from scripts/tools/benchmark_swiglu.py rename to csrc/bench/benchmark_swiglu.py diff --git a/docs/developer/cuda_kernels.md b/docs/developer/cuda_kernels.md index 7b5f7ed..7704938 100644 --- a/docs/developer/cuda_kernels.md +++ b/docs/developer/cuda_kernels.md @@ -84,7 +84,7 @@ OPT 1.3B M=1 is +4.54%. Qwen2 and LLaMA 3 70B M=1, and all three new families at M=8, remain exact PyTorch fallbacks. These are synthetic projection-chain measurements, not whole-model throughput -claims. Reproduce them with `scripts/tools/benchmark_gemv_common.py`. +claims. Reproduce them with `csrc/bench/benchmark_gemv_common.py`. The AstrAI 1B A→B→B→A results use the real `InferenceEngine`, including scheduler, sampling, and CUDA Graph. M=8 stays on PyTorch because its remaining diff --git a/docs/developer/decode_linear_benchmark.md b/docs/developer/decode_linear_benchmark.md index 7119a9c..7af4746 100644 --- a/docs/developer/decode_linear_benchmark.md +++ b/docs/developer/decode_linear_benchmark.md @@ -1,6 +1,6 @@ # Decode linear shape benchmark -`scripts/tools/benchmark_gemv.py` records the `F.linear` baseline used to decide +`csrc/bench/benchmark_gemv.py` records the `F.linear` baseline used to decide whether a BF16 GEMV or small-M kernel should enter automatic inference dispatch. It does not change model execution or select a custom kernel. @@ -10,7 +10,7 @@ modes. Results include device-event latency samples, p50/p90/p99, estimated effective IO bandwidth, and CUDA kernel launches per call. ```bash -CUDA_VISIBLE_DEVICES=0 python scripts/tools/benchmark_gemv.py \ +CUDA_VISIBLE_DEVICES=0 python csrc/bench/benchmark_gemv.py \ --output results/decode_linear.json \ --markdown-output results/decode_linear.md ``` @@ -25,7 +25,7 @@ For direct A/B coverage of the custom kernel and guarded dispatcher across traditional LLaMA and GPT-NeoX decode shapes, use: ```bash -CUDA_VISIBLE_DEVICES=0 PYTHONPATH=. python scripts/tools/benchmark_gemv_common.py \ +CUDA_VISIBLE_DEVICES=0 PYTHONPATH=. python csrc/bench/benchmark_gemv_common.py \ --suite all --family traditional --m 2 4 \ --output results/gemv_common.json ``` diff --git a/docs/developer/swiglu_benchmark.md b/docs/developer/swiglu_benchmark.md index 575ad81..199fc8e 100644 --- a/docs/developer/swiglu_benchmark.md +++ b/docs/developer/swiglu_benchmark.md @@ -1,12 +1,12 @@ # Fused SwiGLU benchmark -`scripts/tools/benchmark_swiglu.py` compares the directly callable fused BF16 +`csrc/bench/benchmark_swiglu.py` compares the directly callable fused BF16 SwiGLU primitive with both `F.linear` and the existing two-GEMV chain. It covers the native AstrAI 1B MLP plus LLaMA 2 7B/13B, LLaMA 3 8B, and GPT-NeoX 20B up/gate shapes at M=1/2/4/8 in eager and CUDA Graph modes. ```bash -CUDA_VISIBLE_DEVICES=0 python scripts/tools/benchmark_swiglu.py \ +CUDA_VISIBLE_DEVICES=0 python csrc/bench/benchmark_swiglu.py \ --output results/swiglu.json \ --markdown-output results/swiglu.md \ --m-values 1,2,4,8 --mode both \