- Muon for 2D matrix params, AdamW for 1D (norm/bias/embed) - MuonMix wrapper handles combined step/zero_grad/state_dict - New CLI args: weight_decay, muon_momentum, muon_nesterov, muon_ns_steps, muon_adjust_lr - Removed adamw_beta1/adamw_beta2/adamw_weight_decay - Moved optimizer/strategy params from signature to **kwargs |
||
|---|---|---|
| .. | ||
| benchmark.py | ||
| generate.py | ||
| preprocess.py | ||
| server.py | ||
| train.py | ||