doc: add SFT length filter (15-token floor) with IFD bias rationale
Per-field length filtering in SFT drops instruction--response pairs with responses shorter than 15 tokens. Short replies exhibit high per-token variance in both conditional and unconditional loss (Appendix A.3 / Figure 5), which would distort downstream IFD-based difficulty estimates.
This commit is contained in:
@@ -154,8 +154,14 @@ pipeline proceeds as follows:
|
||||
previously kept sample $\mathbf{s}'$.
|
||||
\end{enumerate}
|
||||
|
||||
An optional LLM-as-Judge scoring module provides multi-dimensional
|
||||
quality scores that can be used to filter low-quality samples.
|
||||
In addition to deduplication, instruction--response pairs whose
|
||||
response contains fewer than 15 tokens are discarded.
|
||||
Short replies exhibit high per-token variance in both
|
||||
conditional and unconditional loss (Appendix~\ref{sec:ifd_bias},
|
||||
Figure~\ref{fig:length_bias}), which would otherwise distort
|
||||
the IFD-based difficulty estimates used for downstream sample
|
||||
selection. This length floor is applied per-field, analogous to
|
||||
the pretraining filter described above.
|
||||
|
||||
\subsection{DPO Data Generation}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user