ViperEkura

ViperEkura pushed tag v1.3.9 to ViperEkura/AstrAI

2026-08-01 09:23:11 +08:00

ViperEkura pushed tag v1.3.7 to ViperEkura/AstrAI

2026-08-01 09:23:11 +08:00

ViperEkura pushed tag v1.3.12 to ViperEkura/AstrAI

2026-08-01 09:23:10 +08:00

ViperEkura pushed tag v1.3.11 to ViperEkura/AstrAI

2026-08-01 09:23:10 +08:00

ViperEkura pushed tag v1.3.10 to ViperEkura/AstrAI

2026-08-01 09:23:10 +08:00

ViperEkura pushed to main at ViperEkura/AstrAI

2026-08-01 09:23:10 +08:00

3639b50b4a chore: bump version to 1.3.12

d855c09cf3 fix: use torch.optim.AdamW in ManoAdamW instead of NAdamW

d6bfb09863 feat: add grad_snr metric with EMA-based gradient SNR tracking

6db276f37a feat: add Mano manifold optimizer (mano_adamw)

6c76c16480 feat: group train CLI options in --help output

Compare 10 commits »

ViperEkura pushed to main at ViperEkura/AstrAI

2026-07-31 21:05:08 +08:00

21ddead238 fix: stabilize paged decode attention kernels

7aa5ed09d9 refactor: unify rotary embedding interface and update docs

Compare 2 commits »

ViperEkura pushed to main at ViperEkura/AstrAI

2026-07-31 16:24:14 +08:00

75411ce0cc fix: skip CUDA rotary kernel when grad is enabled

9f83d982ec refactor: move compiled kernel .so files into extension/lib

3e67b4f88d perf: add fused CUDA rotary embedding kernel

50cfd0d555 perf: reduce decode overhead in scheduler and executor

5756054d38 build: parametrize CUDA version for wheels and docker

Compare 5 commits »

ViperEkura pushed to main at ViperEkura/DataPipeline

2026-07-31 10:04:12 +08:00

cc451e5492 feat: add Ultra-FineWeb-L3 QA Synthetic pre-train export scripts (en/zh)

ViperEkura pushed to main at ViperEkura/DataPipeline

2026-07-31 08:51:35 +08:00

2b3bf442e9 feat: add SFT process_batch for parallel tokenization + short QA filter script

545104ba70 feat: add Alpaca-GPT4-ZH SFT export script

e999629098 移除缓存脚本中的 data_files 和 max_chunks 限制参数

33c8720d69 feat: 并行 batch tokenization + cache_jsonl 批处理支持

e6787a2036 feat: add BELLE SFT export script, remove Firefly-1.1M-Rephrased

Compare 10 commits »

ViperEkura created branch main in ViperEkura/DataPipeline

2026-07-31 08:51:34 +08:00

ViperEkura pushed to main at ViperEkura/AstrAI

2026-07-31 08:40:56 +08:00

738cb8f128 fix: broadcast ref/old model state_dict for FSDP

28d1bd07cf style: unify decode expf to __expf

02625739fe perf: increase eval batch sizes and add max_seq_len

f688cd9c5a fix: update benchmark to use checkpoint loading and CudaBackend

8055027df7 perf: enable paged MMA kernel for page_size=1

Compare 43 commits »

ViperEkura pushed to master at ViperEkura/AstrAI-paper

2026-07-24 06:38:11 +08:00

e6be33aa53 doc: add SFT length-filter rationale — high PPL + high variance from A.3 IFD figure

ViperEkura pushed to master at ViperEkura/AstrAI-paper

2026-07-24 06:29:40 +08:00

0a1d0573ae doc: add SFT length filter (15-token floor) with IFD bias rationale

ViperEkura pushed to master at ViperEkura/AstrAI-paper

2026-07-24 06:22:03 +08:00

0c2bc916f2 fix: loss_compare caption token count (20B -> 5B)

4e70e827ff remove: drop ckpt_weight_density_per_run.png figure

Compare 2 commits »

ViperEkura pushed to master at ViperEkura/AstrAI-paper

2026-07-24 06:16:17 +08:00

a7bbc7b29f fix: align SFT/DPO figures and text with actual training data

ViperEkura pushed to main at ViperEkura/AstrAI

2026-07-23 18:43:29 +08:00

99b5d2b2da perf: batch tokenizer preprocessing

021e6f3788 style: apply ruff formatting to FSDP2 changes

4e38183e86 fix: make FSDP2 executor work with ABC+Generic model hierarchy

4eeb23e2b3 fix: use copy-on-write mmap mode to silence non-writable tensor warning

ef8783b7e3 fix: separate attn_mask and loss_mask in get_logprobs, compose causal masking in strategy

Compare 31 commits »

ViperEkura pushed to master at ViperEkura/AstrAI-paper

2026-07-20 11:59:19 +08:00

4b37f289c0 fix: attribute muon-25bt larger std to Muon characteristic, not training duration alone

a83555f326 fix: align conclusion, abstract, introduction with body data and full paper scope

Compare 2 commits »

ViperEkura pushed to master at ViperEkura/AstrAI-paper

2026-07-20 11:27:47 +08:00

d93ff48320 docs: fix storage backends (2->3) and per-field length filter

ViperEkura pushed to master at ViperEkura/AstrAI-paper

2026-07-20 04:00:06 +08:00

da0f536526 Update DPO iteration count: 1,500 -> 3,000