diff --git a/main.tex b/main.tex index 9ff0d82..01e487f 100644 --- a/main.tex +++ b/main.tex @@ -154,8 +154,14 @@ pipeline proceeds as follows: previously kept sample $\mathbf{s}'$. \end{enumerate} -An optional LLM-as-Judge scoring module provides multi-dimensional -quality scores that can be used to filter low-quality samples. +In addition to deduplication, instruction--response pairs whose +response contains fewer than 15 tokens are discarded. +Short replies exhibit high per-token variance in both +conditional and unconditional loss (Appendix~\ref{sec:ifd_bias}, +Figure~\ref{fig:length_bias}), which would otherwise distort +the IFD-based difficulty estimates used for downstream sample +selection. This length floor is applied per-field, analogous to +the pretraining filter described above. \subsection{DPO Data Generation}