-
bfc8ff6098
Fix training budget (19k steps / ~20B tokens), add Muon optimizer, correct variance scaling math, move figures to §Training Config, tighten abstract
master
ViperEkura
2026-07-08 22:49:43 +0800
-
4a143b056d
Tighten abstract, add Muon optimizer mention, update ckpt_comparison figure
ViperEkura
2026-07-08 22:27:33 +0800
-
5726bf6c84
Add optimizer/initialization comparison figure (ckpt_comparison)
ViperEkura
2026-07-08 21:59:56 +0800
-
ff13a45d95
Restructure data pipeline: replace IFD analysis with length bias appendix
ViperEkura
2026-07-05 18:59:29 +0800
-
2d52a00149
Update ifd_loss_ratio_density.png
ViperEkura
2026-07-05 12:38:43 +0800
-
f2c2b159ba
Add IFD loss ratio density analysis, weight distribution appendix, and rewrite abstract
ViperEkura
2026-07-05 12:31:40 +0800
-
459523f0e4
Fix IFD citation: correct author list and venue (NAACL 2024, not NeurIPS)
ViperEkura
2026-07-05 00:24:43 +0800
-
4a7bf7ac3f
Add GQA/SwiGLU/RoPE formula blocks, Alpaca-GPT4 citation, and improve IFD-Loss Ratio interpretation
ViperEkura
2026-07-05 00:17:09 +0800
-
5c915514b3
Add correlation analysis interpretation to IFD bias section
ViperEkura
2026-07-04 23:45:55 +0800
-
bc8d4066c1
Add IFD analysis: density/length-bias figures and appendix
ViperEkura
2026-07-04 23:33:12 +0800
-
9534316bac
Fix duplicate paragraph and misleading table caption
ViperEkura
2026-07-04 22:39:16 +0800
-
b40fcc86c7
Refine abstract, switch to Times font, remove date, fix overfull hboxes with microtype
ViperEkura
2026-07-04 22:31:45 +0800
-
dd412b3c9c
Add IFD analysis, appendix, and improve abstract
ViperEkura
2026-07-04 22:25:18 +0800
-
73cb02f0aa
fix training hyperparams to match pretraining script: lr 1.5e-4, wd 0.1, warmup 0.02
ViperEkura
2026-06-28 16:57:30 +0800
-
adc284920a
add Alembic SFT data cleaning section with MinHash algorithm and fix bibliography order
ViperEkura
2026-06-28 16:52:11 +0800
-
13089b002a
fix training config: grad_accum 32, remove unused validation & activation ckpt
ViperEkura
2026-06-28 15:35:15 +0800
-
d3fbe0ecc6
refactor: numerical stability analysis with residual scaling comparison
ViperEkura
2026-06-28 14:36:12 +0800
-
3db1096c78
fix: add caption package, fix overfull hbox, correct RoFormer author names
ViperEkura
2026-06-25 16:53:26 +0800
-
6f9278b35b
Initial commit: BF16 weight lock-in paper for 1.2B Transformer training with AstrAI
ViperEkura
2026-06-25 15:51:05 +0800