ViperEkura
/
AstrAI
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
481 Commits 1 Branch 9 Tags 3.9 MiB
Commit Graph

481 Commits

Author SHA1 Message Date
ViperEkura 9ebaea840f perf: cp.async K/V loads, shared sQ staging, causal skip, XOR swizzle 
- cp.async global→shared for K/V full-tile loads, eliminates 99.6% of shared-store bank conflicts (612K→2.7K per ncu)
- add cp_async_16/commit/wait_all/wait_group<N> helpers in mma utils
- shared sQ staging (single area, serialized per-warp load), cuts smem from (2*BC + WARPS*BR)*LD to (2*BC + BR)*LD bf16
- pre-scale Q by attention scale during Q load, removes per-tile scale multiply in softmax loop
- causal tile skipping: block-level early break + warp-level skip
- scalar fallback only for last partial tile
- XOR swizzle (swiz_col) at 8-bf16 chunk granularity, eliminates ldmatrix bank conflicts without LD padding, LD=HEAD_DIM (zero smem waste), saves 1280 bytes/block vs HEAD_DIM+8 padding
 2026-07-08 12:30:32 +08:00
ViperEkura 6adc221c10 refactor: extract shared MMA utils into gqa_mma_utils.cuh 
- Move mma16816, ld2, pk2, pkb, ldmatrix_x4/x2/x2_trans to shared header
- gqa_prefill_attn_mma.cuh and gqa_decode_attn_mma.cuh both include it
 2026-07-07 23:01:15 +08:00
ViperEkura 9e63cb9ed0 feat: MMA head-packing decode kernel with scalar fallback dispatch 
- Add gqa_decode_attn_mma.cuh for tensor-core decode path
- Add dispatch_decode<> selecting MMA vs scalar based on G and mask
- Add TORCH_CHECK for unsupported head_dim instead of silent scalar launch
 2026-07-07 22:56:02 +08:00
ViperEkura 4225518cf3 perf: add fast-math and vectorization nvcc/cxx build flags 
- centralize CXX_FLAGS/NVCC_FLAGS in csrc/build.py as single source
- add --use_fast_math, --ptxas-options=-O3,-v, --extra-device-vectorization
- add -march=native -funroll-loops host flags
- setup.py reads shared cxx_flags/nvcc_flags from registry
- sync pure-C test build commands with new flags
 2026-07-07 22:28:32 +08:00
ViperEkura c50adbaac0 feat : replace AdamW with MuonMix (Muon + AdamW) optimizer 
- Muon for 2D matrix params, AdamW for 1D (norm/bias/embed)
- MuonMix wrapper handles combined step/zero_grad/state_dict
- New CLI args: weight_decay, muon_momentum, muon_nesterov, muon_ns_steps, muon_adjust_lr
- Removed adamw_beta1/adamw_beta2/adamw_weight_decay
- Moved optimizer/strategy params from signature to **kwargs
 2026-07-07 14:10:36 +08:00
ViperEkura 536dbc0c9a fix: set tqdm postfix before update so first step shows metrics 2026-07-07 00:14:13 +08:00
ViperEkura 4af7acd449 fix: support single .h5 file loading in load_h5 2026-07-07 00:11:16 +08:00
ViperEkura 53ed52b4b8 refactor: extension dispatch layer with CUDA/torch fallback 
- Add gqa_decode_attn/gqa_prefill_attn dispatch functions
- Internal _available/__modules with underscore prefix
- CUDA kernel path with F.scaled_dot_product_attention fallback
- GQA head expansion in fallback path
 2026-07-06 21:07:16 +08:00
ViperEkura f1cc7cedce feat: ldmatrix + smem padding for mma prefill kernel 
- replace scalar fragment loads with ldmatrix.sync.x4/x2
- add smem row-stride padding (LD = HEAD_DIM + 8) to eliminate 8-way bank conflicts from HEAD_DIM being a 32-bank multiple
- switch build flag from positive to negative: -DASTRAI_NO_MMA for pre-sm_80 only; mma is the default path
- vectorize scalar path smem loads with float4 ld8
- fix pure-C test configs for ld8 alignment
 2026-07-06 20:55:22 +08:00
ViperEkura ddc4bd1cf6 feat: tensor-core mma prefill with build-time dispatch 
- add register-resident flash-attention kernel using mma.sync.m16n8k16
- dispatch mma vs scalar at build time: pre-sm_80 defines
  -DASTRAI_NO_MMA, else defaults to mma
- scalar path vectorized with float4 smem loads (ld8)
 2026-07-06 20:33:24 +08:00
ViperEkura cc36530c73 perf: group-split register-blocking gqa_prefill kernel 
- one query row per group of G=8 lanes, each owning HEAD_DIM/G dims of qreg[]/acc[] in registers
- removes full 32-lane warp_reduce_sum; S dot reduces over only G lanes
- templated on <HEAD_DIM,G,ROWS,P_BC>, block=(G,ROWS)=(8,32)
- per-group shuffle mask so causal loop-bound divergence doesn't deadlock the shuffle
- update pure-C test to the templated launch
 2026-07-06 18:33:08 +08:00
ViperEkura 11fa807cfc fix: correct prefill mask index, unify GQA kernel interface 
- Fix mask indexing: batch*q_len*kv_len -> batch*kv_len
- Add csrc/kernels/gqa_common.cuh with shared GQAParams struct
- Unify decode/prefill Python API: both accept (q,k,v,mask=None,...)
- Decode now supports optional mask, is_causal, causal_offset, scale
- Rename struct fields: B->batch, Hq->q_head, Hk->kv_head, D->head_dim
- Use py::arg() for correct None/defaults handling in pybind11
- Update pure C tests and build instructions (-arch=sm_89)
 2026-07-06 17:21:23 +08:00
ViperEkura bcdd93e0eb feat: split kernel defs from bindings, add prefill tiled kernel and pure C tests 
- Split .cuh/.cu for gqa_decode_attn and gqa_prefill_attn
- gqa_prefill_attn: tiled shared-memory K/V, fused load, compute-opt, mask support
- Add pure C tests under csrc/tests/ for fast nvcc-only iteration
- Update .gitignore for build artifacts
 2026-07-06 16:14:55 +08:00
ViperEkura 579b8c3129 fix: correct gqa_decode_attn reduction + add gqa_prefill_attn 
- gqa_decode_attn: rewrite to per-KV-head, K in smem
- gqa_prefill_attn: new kernel for Q_len > 1 with GQA
 2026-07-06 13:45:18 +08:00
ViperEkura d7da51569f docs: update install instructions in EN/CN README 2026-07-06 12:25:36 +08:00
ViperEkura e8e228d035 feat: add optional CUDA kernel system (csrc/) + fused GQA decode attention 
Structure:
  csrc/               -- .cu sources + build.py registry
  astrai/extension/   -- compiled .so + __init__.py (import dispatcher)
  setup.py            -- CUDAExtension from csrc/build.py REGISTRY

Control: CSRC_KERNELS=true|false env var at install time.
Fallback: astrai.extension.available dict for runtime detection.
 2026-07-06 12:09:58 +08:00
ViperEkura 2579658e15 chore : shields release badge from /release to /tag 2026-07-05 20:34:40 +08:00
ViperEkura f0cd0134c6 fix : update benchmark for v1.3.8 cache API, add argparse and cache type switch 
- Replaced old KVCage API with PageCache/ContiguousCache
- Added --cache contiguous|paged switch for decoding comparison
- Added argparse for all params (batch/prompt/gen/device/dtype)
- Fixed PageCache decode crash by extending pages for full sequence
 2026-07-05 20:30:26 +08:00
ViperEkura abb96996f8 docs : sync 6 doc files to actual code 
- architecture.md: removed TrainConfig.log_interval, split KVCache into
  PageCache/ContiguousCache with CacheView/PageCacheView/ContiguousCacheView,
  added JsonlStore, fixed GradientCheckpointingCallback type,
  CheckpointCallback typo, ProgressBarCallback hooks
- training.md: added position_ids to SFT keys, fixed callback hook table,
  removed merged ValidationCallback
- inference.md: documented ContiguousCache default vs PageCache paged
- dataflow.md: added JsonlStore to storage backends and format detection
- params.md: removed nonexistent --log_interval
- preprocessing.md: updated timestamp
 2026-07-05 19:35:18 +08:00
ViperEkura bbe6ff2d8f release : v1.3.8 
- refactor: 重写 IFD 评估为三层架构，引入 BFD 装箱与自定义 attention mask 批处理打分
- refactor: 重写 HumanEval 评估为函数式流水线，修复测试超时与动态 pass@k
- perf: 替换 paged KV cache 为 ContiguousCache，解码所有 group
- feat: 新增 ROUGE 评估脚本、JSONL 数据集 store、stream_chat 参数
- fix: 修复 IFD token-set 不对称、SFT position_ids 默认值、文档边界保留
 2026-07-05 19:12:33 +08:00
ViperEkura db9b39b084 fix: resolve IFD token-set asymmetry and support single-token answers 
- Sentinel-anchored unconditional pass: both branches now predict the same N response tokens
- Single-token responses (rl=1) fully supported
- ctx_len tracked per sample; skip_reason replaces silent None
- --per_token flag for per-token IFD breakdown
 2026-07-05 17:48:26 +08:00
ViperEkura 849e1e00a3 refactor: clean up inference design patterns 
1. KVCache base: add default task_cached/task_record_hashes, remove getattr from scheduler
2. Remove page_size param from scheduler constructor (ContiguousCache-only)
3. InferenceEngine expose cache param for KVCache injection
4. Rename page_cache -> kv_cache in Executor
5. Move stream_callback from Task to TaskManager._callbacks dict
6. TaskManager.clear_queues clears callbacks
 2026-07-05 11:41:54 +08:00
ViperEkura 5416c2e8fb perf: replace paged KV cache with contiguous ContiguousCache, decode all groups 
- Add KVCache/CacheView abstract base classes in cache.py
- Add ContiguousCache (contiguous per-slot buffer, default) alongside PageCache (paged, renamed from old KVCache)
- Merge make_table_tensor + bind into bind_tasks on KVCache interface
- Remove task_cached/task_record_hashes from base class (PageCache-only)
- Scheduler: decode all position groups instead of just the largest (eliminates 63% group skip rate)
- Scheduler: accept optional cache param for swapping implementations
- Model layer type hints use CacheView base class
- Batch 1-32: 1-7% speedup from eliminating Storage.gather overhead
- All 183 inference tests pass
 2026-07-05 11:34:36 +08:00
ViperEkura 599a51f4f7 fix: reliable test timeout, separate generate/test phases, dynamic pass@k 
- Replace SIGALRM+exec() with subprocess.run(timeout=) for test execution
- Add --test_only flag to skip generation and test existing completions
- Add --generate_only flag for generation-only runs
- Derive pass@k values from num_samples (filter k > n)
- Support loading completions from array JSON (not just JSONL)
 2026-07-05 08:47:30 +08:00
ViperEkura 17d6eaa2f2 refactor: rewrite humaneval evaluation with functional pipeline design 
- fix KeyError race condition in inference cache touch()
- EvalConfig dataclass for centralized configuration
- load->generate->extract->test->score->report pipeline
- two-phase generation+testing for max GPU utilization
- signal-based SIGALRM timeout protection for code exec
- suppress subprocess stdout/stderr pollution
 2026-07-05 07:58:28 +08:00
ViperEkura 2d908639e9 feat : add ROUGE evaluation script (manual impl, no deps) 
- ROUGE-1/2 via n-gram overlap (Counter)
- ROUGE-L via LCS (DP)
- CLI: python scripts/eval/evaluate_rouge.py --data_path ... --output ...
- Library: compute_rouge(ref, cand) -> dict of precision/recall/f1
 2026-07-05 01:15:01 +08:00
ViperEkura c7158418dd perf: add BFD bin-packing and custom attention mask to IFD batch scoring 2026-07-04 18:58:13 +08:00
ViperEkura 4d3c9341c1 refactor: rewrite IFD evaluation with clean three-layer architecture 2026-07-04 18:33:51 +08:00
ViperEkura 4e508afa2d fix : SFT pipeline position_ids default & doc boundary preservation 
- change position_ids_mode default from "none" to "doc_reset" so SFT preprocessing always generates position_ids (was causing dataset load KeyError)
- generate per-doc position_ids before packing (doc_reset mode), preserving document boundaries for BFD packing (cross-doc attention leak fix)
- change _align_bucket padding from [1] to [0] to avoid accidentally training on loss_mask padding
 2026-07-04 15:59:11 +08:00
ViperEkura 8999ca89b8 feat: add JSONL dataset store with on-the-fly tokenization 
- Add JsonlStore registered under "jsonl" in astrai/dataset/storage.py
- Reuse PipelineConfig schema for JSONL dataset configuration
- Update detect_format to recognize JSONL directories and files
- Move save_h5/load_h5/save_bin/load_bin to astrai/serialization
- Split astrai/serialization.py into checkpoint/dataset submodules
- Add tests for JSONL detection, seq/SFT stores, and config roundtrip
 2026-07-04 15:42:33 +08:00
ViperEkura 1adca39cd8 fix: handle long sequences and optimize IFD computation 2026-07-04 08:35:45 +08:00
ViperEkura 204873fa2f fix: handle long sequences and optimize IFD computation 2026-07-04 08:23:32 +08:00
ViperEkura a5c1de6b1b feat: add model_path temperature top_p top_k max_tokens system_prompt args to stream_chat 2026-07-04 07:33:32 +08:00
ViperEkura 27524ad085 fix: reset sampler iter at epoch end so progress bar shows total after first epoch 2026-07-04 06:35:55 +08:00
ViperEkura 27d1921d9c fix: scheduler division-by-zero, loss_mask bool 
- schedule.py: guard warmup_steps/lr_decay_steps against zero
- strategy.py: use ~loss_mask instead of loss_mask==0 on bool tensor
 2026-07-03 22:04:55 +08:00
ViperEkura 70c0e5de90 refactor: merge validation into MetricCallback, simplify progress bar to optimizer steps 
- Remove separate ValidationCallback, merge into MetricCallback
- Progress bar now tracks optimizer steps instead of micro-steps
- Remove unused log_interval config field and CLI flag
- Fix validation all_reduce: use SUM(loss, count) instead of AVG
- Simplify metric logging: always log every optimizer step
- Add grad_norm display to progress bar
 2026-07-03 21:43:08 +08:00
ViperEkura dfb151537b fix: ForwardRef._evaluate Python 3.12 compatibility 2026-07-03 18:41:19 +08:00
ViperEkura 500c605fad fix: unify scheduler min_rate default to 0.01, clamp WSD warmup 2026-07-03 17:52:23 +08:00
ViperEkura dc9faca3b1 fix: align docs with actual code (40+ inconsistencies) 
- Remove nonexistent Muon class from architecture diagram
- Fix Checkpoint/TrainConfig/TrainContext field names (iteration -> consumed_samples, start_batch -> start_samples)
- Add missing fields: neftune_alpha, val_split, grad_norm, optimizer_step, tool_calls/tools
- Fix CLI param defaults: --log_interval 1, --metrics [loss,lr,grad_norm], --start_samples
- Add missing scheduler CLI params; remove nonexistent --num_workers from preprocess docs
- Fix inference SSE format, stats response keys, error codes to match actual server output
- Fix preprocessing docs: BOS once, shard_0000 layout, from_json->from_file, GRPO prompts_mask
- Fix dataflow detect_format/_normalize descriptions; correct callback order in training.md
 2026-06-30 20:47:23 +08:00
ViperEkura aabb0d83e9 refactor : replace iteration with consumed_samples 
- Replace context.iteration with consumed_samples (global sample count)
- Add optimizer_step property derived from consumed_samples
- Checkpoint meta.json stores consumed_samples, drops iteration
- CLI --start_batch renamed to --start_samples (per-rank samples)
- Checkpoint dir naming: epoch_X_step_Y instead of epoch_X_iter_Y
- Metric log entries use step and consumed_samples fields
- Backward compat removed (old iteration checkpoints unsupported)
 2026-06-30 18:42:42 +08:00
ViperEkura 44579ea6dc refactor : metric 日志改为以 optimizer step 为单位，默认每步记录 
- log_interval 默认 100 -> 1，语义从 batch iteration 改为 optimizer step
- step 指标从 on_batch_end 移到 on_optimizer_step，不受梯度累积影响
- JSONL 条目新增 step 字段，保留 iter
- flush 落盘仍在 on_batch_end
 2026-06-30 15:12:31 +08:00
ViperEkura 0f1fcb079f refactor : grad_norm 指标简化，clip_grad_norm 移至 executor 
- metrics 默认加入 grad_norm，移除 grad_std/max/min/mean/nan_num
- grad_norm 默认返回总 L2 范数，per_param=True 返回各参数范数
- clip_grad_norm 从 callback 移至 BaseExecutor/FSDPExecutor
- FSDPExecutor 覆盖为 model.clip_grad_norm_() 保证分布式正确
- ctx_get_grad_norm 改为读取 context.grad_norm
 2026-06-30 14:59:43 +08:00
ViperEkura 84d4769163 feat: SVD 有效秩/权重统计分析脚本 2026-06-29 21:39:22 +08:00
ViperEkura bf09a35c95 feat: optimizer 参数分组，bias/norm 不做 weight decay 2026-06-27 16:30:34 +08:00
ViperEkura 6715461a36 chore : 升级 torch 2.11.0+cu128，移除自定义 Muon，修复 gloo device_id 
- torch 2.7.1-cu126 升级至 2.11.0-cu128，numpy 2.3.2 升级至 2.4.4
- 移除 astrai/trainer/optim.py，改用 torch.optim.Muon
- parallel setup: gloo 后端不再传递 device_id，单卡多进程不再报错
 2026-06-27 16:10:37 +08:00
ViperEkura b4587c5d08 refactor : metric_logger 改用事件类型 (type=step/validation/epoch) 
- 每种事件独立 schema，不再混入 null 字段
- 回调顺序 validation 移到 metric_logger 之前，确保 on_optimizer_step 先跑
- 用内部 _last_val_loss 代替 TrainContext.last_val_iter 判断新验证
- 修复 factory.py 未使用导入、evaluate_ifeval.py 多余 f 前缀
 2026-06-25 17:18:20 +08:00
ViperEkura 88ec63121d feat : GPT-2 residual scaling weight init 
- Linear: normal(0, init_std) replaces kaiming_uniform_(a=sqrt(5))
- o_proj / mlp.down: init_std = 0.02 / sqrt(2 * n_layers)
- MoE: expert down scaled by 1/sqrt(1/n_shared + 1/K)
- Embedding: normal(0, 0.02), unchanged
 2026-06-25 15:08:31 +08:00
ViperEkura 01d2da2893 feat : 训练支持 --schedule_type 及对应调度器参数 
- --schedule_type 可选 cosine/sgdr/wsd，默认 cosine
- --min_rate 统一控制最小 LR 比率
- --cycle_length / --t_mult 用于 sgdr
- --stable_steps / --decay_steps 用于 wsd，自动计算默认值
 2026-06-22 10:35:56 +08:00
ViperEkura 25d4ea3f91 refactor : 压缩测试代码，消除重复 
- fixture 替代重复实例化和 tokenizer 落盘
- parametrize 合并同构测试
- helper 消除 save_h5 + DatasetFactory.load 样板
- 净减 272 行
 2026-06-19 14:54:39 +08:00
ViperEkura 39985840c7 refactor : neftune_alpha 在 Embedding 构造时传入，由模型配置链路负责 
- BaseModelConfig 添加 neftune_alpha 字段 (默认 0.0)
- Embedding.__init__ 接受 neftune_alpha 参数，不再外部 set
- AutoRegressiveLM / EmbeddingEncoder 从 config 传入 neftune_alpha
- train.py 将 CLI 参数注入 config 后再创建模型
- TrainContextBuilder 移除 neftune 设置（不再是其职责）
 2026-06-19 14:23:27 +08:00