diff --git a/main.tex b/main.tex index 8488662..dc1be88 100644 --- a/main.tex +++ b/main.tex @@ -364,7 +364,7 @@ We use $\beta=0.1$, a batch of 64 preference pairs per step, and the same hybrid Muon/AdamW optimizer. Unlike the pretraining and SFT phases, which both employ WSD (warmup--stable--decay) scheduling to maintain a long high-learning-rate plateau, DPO alignment uses a -cosine schedule with short linear warmup. The shorter ~1{,}500-step +cosine schedule with short linear warmup. The shorter ~3{,}000-step alignment run does not benefit from an extended stable phase; instead, cosine decay lowers the learning rate steadily, which discourages over-optimisation away from the reference distribution and matches the @@ -373,7 +373,7 @@ observed convergence pattern in Figure~\ref{fig:dpo_metrics}. \begin{figure}[H] \centering \includegraphics[width=0.95\linewidth]{data/dpo_metrics.png} -\caption{DPO training metrics over $\sim$1{,}500 alignment steps: +\caption{DPO training metrics over $\sim$3{,}000 alignment steps: preference loss (raw and 100-step moving average), learning rate schedule, and gradient norm.} \label{fig:dpo_metrics}