doc: add SFT length-filter rationale — high PPL + high variance from A.3 IFD figure
- Add 15-token length floor to SFT samples, with explicit reference to Appendix A.3 / Figure 5 (ifd_length_grid) - Short replies (<10 tokens) show both high per-token perplexity (L_uncond ~6-8, PPL ~400-3000 vs long replies ~2-3, PPL ~7-20) and wide variance in L_cond (span 0-17.5), which distorts downstream IFD-based difficulty estimates.
This commit is contained in:
@@ -154,14 +154,16 @@ pipeline proceeds as follows:
|
|||||||
previously kept sample $\mathbf{s}'$.
|
previously kept sample $\mathbf{s}'$.
|
||||||
\end{enumerate}
|
\end{enumerate}
|
||||||
|
|
||||||
In addition to deduplication, instruction--response pairs whose
|
A length filter is applied to SFT samples based on the IFD
|
||||||
response contains fewer than 15 tokens are discarded.
|
length-bias analysis in Appendix~\ref{sec:ifd_bias}
|
||||||
Short replies exhibit high per-token variance in both
|
(Figure~\ref{fig:length_bias}): instruction--response pairs
|
||||||
conditional and unconditional loss (Appendix~\ref{sec:ifd_bias},
|
whose response contains fewer than 15 tokens are discarded,
|
||||||
Figure~\ref{fig:length_bias}), which would otherwise distort
|
because short replies exhibit both high per-token perplexity
|
||||||
the IFD-based difficulty estimates used for downstream sample
|
($L_{\text{uncond}} \approx 6\text{--}8$, PPL~$\approx 400\text{--}3000$)
|
||||||
selection. This length floor is applied per-field, analogous to
|
and wide variance in both $L_{\text{cond}}$ and $L_{\text{uncond}}$,
|
||||||
the pretraining filter described above.
|
which would distort downstream IFD-based difficulty estimates.
|
||||||
|
The threshold is applied per-field, analogous to the pretraining
|
||||||
|
filter described above.
|
||||||
|
|
||||||
\subsection{DPO Data Generation}
|
\subsection{DPO Data Generation}
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user