Add effective rank analysis, weight distribution evolution, Muon/SFT training figures

Browse Source

- Add SVD effective rank (ER@99%) and condition number analysis across
  three checkpoints (kami-15bt, norm-15bt, muon-25bt) in appendix
- Add post-training weight distribution analysis as §5.4, verifying
  residual scaling persists throughout training
- Add per-component weight std table in appendix showing Muon constrains
  early-stage drift vs Normal init
- Add Muon optimizer training dynamics figure (muon_pt.png)
- Add SFT training metrics figure (sft_metrics.png) on mixed CN-EN data
- Restructure: move effective rank to appendix, weight distribution to
  §5.4, delete standalone §6, tighten abstract to prose-only
- Update conclusion to reference new analyses

...

This commit is contained in:

ViperEkura

2026-07-17 23:05:21 +08:00

parent bfc8ff6098

commit a0c39601a0

5 changed files with 217 additions and 36 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

data/ckpt_weight_density.png

BIN

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 940 KiB

data/ckpt_weight_density_per_run.png

BIN

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 315 KiB

data/muon_pt.png

BIN

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 128 KiB

data/sft_metrics.png

BIN

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 176 KiB