Compare commits

...
4 Commits
Author SHA1 Message Date
ViperEkura e6be33aa53 doc: add SFT length-filter rationale — high PPL + high variance from A.3 IFD figure
- Add 15-token length floor to SFT samples, with explicit reference
  to Appendix A.3 / Figure 5 (ifd_length_grid)
- Short replies (<10 tokens) show both high per-token perplexity
  (L_uncond ~6-8, PPL ~400-3000 vs long replies ~2-3, PPL ~7-20)
  and wide variance in L_cond (span 0-17.5), which distorts
  downstream IFD-based difficulty estimates.
2026-07-24 06:37:35 +08:00
ViperEkura 0a1d0573ae doc: add SFT length filter (15-token floor) with IFD bias rationale
Per-field length filtering in SFT drops instruction--response pairs
with responses shorter than 15 tokens. Short replies exhibit high
per-token variance in both conditional and unconditional loss
(Appendix A.3 / Figure 5), which would distort downstream IFD-based
difficulty estimates.
2026-07-24 06:27:10 +08:00
ViperEkura 0c2bc916f2 fix: loss_compare caption token count (20B -> 5B)
The loss_compare.png x-axis only spans 0-5B tokens. The caption
incorrectly claimed ~20B tokens.
2026-07-24 06:21:52 +08:00
ViperEkura 4e70e827ff remove: drop ckpt_weight_density_per_run.png figure
The per-run weight density figure contained misleading legacy
iteration labels (500k/1M iter) that contradicted the paper's
stated token budgets (15B). Removing it avoids confusion; the
per-category density plot (ckpt_weight_density.png) and Table 7
remain as the primary evidence.
2026-07-24 06:20:08 +08:00
+11 -14
View File
@@ -154,8 +154,16 @@ pipeline proceeds as follows:
previously kept sample $\mathbf{s}'$. previously kept sample $\mathbf{s}'$.
\end{enumerate} \end{enumerate}
An optional LLM-as-Judge scoring module provides multi-dimensional A length filter is applied to SFT samples based on the IFD
quality scores that can be used to filter low-quality samples. length-bias analysis in Appendix~\ref{sec:ifd_bias}
(Figure~\ref{fig:length_bias}): instruction--response pairs
whose response contains fewer than 15 tokens are discarded,
because short replies exhibit both high per-token perplexity
($L_{\text{uncond}} \approx 6\text{--}8$, PPL~$\approx 400\text{--}3000$)
and wide variance in both $L_{\text{cond}}$ and $L_{\text{uncond}}$,
which would distort downstream IFD-based difficulty estimates.
The threshold is applied per-field, analogous to the pretraining
filter described above.
\subsection{DPO Data Generation} \subsection{DPO Data Generation}
@@ -302,7 +310,7 @@ Sequence length & 2,048 tokens \\
\centering \centering
\includegraphics[width=0.50\linewidth]{data/loss_compare.png} \includegraphics[width=0.50\linewidth]{data/loss_compare.png}
\caption{Training loss curves: GPT-2 residual scaling vs.~Kaiming \caption{Training loss curves: GPT-2 residual scaling vs.~Kaiming
initialization over $\sim$20B tokens.} initialization over $\sim$5B tokens.}
\label{fig:loss} \label{fig:loss}
\end{figure} \end{figure}
@@ -547,17 +555,6 @@ spread.}
\label{fig:ckpt_weight_density} \label{fig:ckpt_weight_density}
\end{figure} \end{figure}
\begin{figure}[H]
\centering
\includegraphics[width=0.95\linewidth]{data/ckpt_weight_density_per_run.png}
\caption{Per-checkpoint weight density breakdowns. The three checkpoints
are \texttt{norm-15bt} (AdamW, Normal init, 15B tokens), \texttt{kami-15bt}
(AdamW, GPT-2 residual scaling, 15B tokens), and \texttt{muon-25bt}
(Muon, 25B tokens). Note: the iteration labels inside the figure reflect
legacy script metadata and should be ignored in favour of the token
budgets stated here.}
\label{fig:ckpt_weight_density_per_run}
\end{figure}
% ====================================================================== % ======================================================================
\section{Conclusion} \section{Conclusion}