ViperEkura/AstrAI-paper
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
20 Commits 1 Branch 0 Tags
a0c39601a03db4a7d528d62de4d688ae24464106
Commit Graph
12 Commits
Author SHA1 Message Date
ViperEkura a0c39601a0 Add effective rank analysis, weight distribution evolution, Muon/SFT training figures 
- Add SVD effective rank (ER@99%) and condition number analysis across
  three checkpoints (kami-15bt, norm-15bt, muon-25bt) in appendix
- Add post-training weight distribution analysis as §5.4, verifying
  residual scaling persists throughout training
- Add per-component weight std table in appendix showing Muon constrains
  early-stage drift vs Normal init
- Add Muon optimizer training dynamics figure (muon_pt.png)
- Add SFT training metrics figure (sft_metrics.png) on mixed CN-EN data
- Restructure: move effective rank to appendix, weight distribution to
  §5.4, delete standalone §6, tighten abstract to prose-only
- Update conclusion to reference new analyses
 2026-07-17 23:05:21 +08:00
ViperEkura bfc8ff6098 Fix training budget (19k steps / ~20B tokens), add Muon optimizer, correct variance scaling math, move figures to §Training Config, tighten abstract 2026-07-08 22:49:43 +08:00
ViperEkura 4a143b056d Tighten abstract, add Muon optimizer mention, update ckpt_comparison figure 2026-07-08 22:27:33 +08:00
ViperEkura 5726bf6c84 Add optimizer/initialization comparison figure (ckpt_comparison) 2026-07-08 21:59:56 +08:00
ViperEkura ff13a45d95 Restructure data pipeline: replace IFD analysis with length bias appendix 
- Remove Section 2.4 (IFD analysis) from main text
- Delete obsolete IFD figures (ifd_compare_clean, ifd_density_dist,
  ifd_both_vs_lossratio, ifd_loss_ratio_density)
- Update ifd_length_grid.png with new data
- Rewrite Appendix A: IFD definition + quantitative summary +
  representative samples table + length bias analysis
- Update abstract to remove IFD findings
- Fix .gitignore to only track .png in data/
- Clean up stale analysis scripts
 2026-07-05 18:59:29 +08:00
ViperEkura 2d52a00149 Update ifd_loss_ratio_density.png 2026-07-05 12:38:43 +08:00
ViperEkura f2c2b159ba Add IFD loss ratio density analysis, weight distribution appendix, and rewrite abstract 2026-07-05 12:31:40 +08:00
ViperEkura 4a7bf7ac3f Add GQA/SwiGLU/RoPE formula blocks, Alpaca-GPT4 citation, and improve IFD-Loss Ratio interpretation 2026-07-05 00:17:09 +08:00
ViperEkura 5c915514b3 Add correlation analysis interpretation to IFD bias section 2026-07-04 23:46:25 +08:00
ViperEkura bc8d4066c1 Add IFD analysis: density/length-bias figures and appendix 2026-07-04 23:33:12 +08:00
ViperEkura dd412b3c9c Add IFD analysis, appendix, and improve abstract 2026-07-04 22:25:18 +08:00
ViperEkura d3fbe0ecc6 refactor: numerical stability analysis with residual scaling comparison 2026-06-28 14:36:12 +08:00