doc: add SFT length-filter rationale — high PPL + high variance from A.3 IFD figure

- Add 15-token length floor to SFT samples, with explicit reference
  to Appendix A.3 / Figure 5 (ifd_length_grid)
- Short replies (<10 tokens) show both high per-token perplexity
  (L_uncond ~6-8, PPL ~400-3000 vs long replies ~2-3, PPL ~7-20)
  and wide variance in L_cond (span 0-17.5), which distorts
  downstream IFD-based difficulty estimates.
This commit is contained in:
2026-07-24 06:37:35 +08:00
parent 0a1d0573ae
commit e6be33aa53
+10 -8
View File
@@ -154,14 +154,16 @@ pipeline proceeds as follows:
previously kept sample $\mathbf{s}'$. previously kept sample $\mathbf{s}'$.
\end{enumerate} \end{enumerate}
In addition to deduplication, instruction--response pairs whose A length filter is applied to SFT samples based on the IFD
response contains fewer than 15 tokens are discarded. length-bias analysis in Appendix~\ref{sec:ifd_bias}
Short replies exhibit high per-token variance in both (Figure~\ref{fig:length_bias}): instruction--response pairs
conditional and unconditional loss (Appendix~\ref{sec:ifd_bias}, whose response contains fewer than 15 tokens are discarded,
Figure~\ref{fig:length_bias}), which would otherwise distort because short replies exhibit both high per-token perplexity
the IFD-based difficulty estimates used for downstream sample ($L_{\text{uncond}} \approx 6\text{--}8$, PPL~$\approx 400\text{--}3000$)
selection. This length floor is applied per-field, analogous to and wide variance in both $L_{\text{cond}}$ and $L_{\text{uncond}}$,
the pretraining filter described above. which would distort downstream IFD-based difficulty estimates.
The threshold is applied per-field, analogous to the pretraining
filter described above.
\subsection{DPO Data Generation} \subsection{DPO Data Generation}