Update DPO iteration count: 1,500 -> 3,000
This commit is contained in:
@@ -364,7 +364,7 @@ We use $\beta=0.1$, a batch of 64 preference pairs per step, and the
|
||||
same hybrid Muon/AdamW optimizer. Unlike the pretraining and SFT
|
||||
phases, which both employ WSD (warmup--stable--decay) scheduling to
|
||||
maintain a long high-learning-rate plateau, DPO alignment uses a
|
||||
cosine schedule with short linear warmup. The shorter ~1{,}500-step
|
||||
cosine schedule with short linear warmup. The shorter ~3{,}000-step
|
||||
alignment run does not benefit from an extended stable phase; instead,
|
||||
cosine decay lowers the learning rate steadily, which discourages
|
||||
over-optimisation away from the reference distribution and matches the
|
||||
@@ -373,7 +373,7 @@ observed convergence pattern in Figure~\ref{fig:dpo_metrics}.
|
||||
\begin{figure}[H]
|
||||
\centering
|
||||
\includegraphics[width=0.95\linewidth]{data/dpo_metrics.png}
|
||||
\caption{DPO training metrics over $\sim$1{,}500 alignment steps:
|
||||
\caption{DPO training metrics over $\sim$3{,}000 alignment steps:
|
||||
preference loss (raw and 100-step moving average), learning rate
|
||||
schedule, and gradient norm.}
|
||||
\label{fig:dpo_metrics}
|
||||
|
||||
Reference in New Issue
Block a user