From e6be33aa5319457568dca762f33482e66ff0ca96 Mon Sep 17 00:00:00 2001
From: ViperEkura <3081035982@qq.com>
Date: Fri, 24 Jul 2026 06:37:35 +0800
Subject: [PATCH] =?UTF-8?q?doc:=20add=20SFT=20length-filter=20rationale=20?=
 =?UTF-8?q?=E2=80=94=20high=20PPL=20+=20high=20variance=20from=20A.3=20IFD?=
 =?UTF-8?q?=20figure?=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

- Add 15-token length floor to SFT samples, with explicit reference
  to Appendix A.3 / Figure 5 (ifd_length_grid)
- Short replies (<10 tokens) show both high per-token perplexity
  (L_uncond ~6-8, PPL ~400-3000 vs long replies ~2-3, PPL ~7-20)
  and wide variance in L_cond (span 0-17.5), which distorts
  downstream IFD-based difficulty estimates.
---
 main.tex | 18 ++++++++++--------
 1 file changed, 10 insertions(+), 8 deletions(-)

diff --git a/main.tex b/main.tex
index 01e487f..d8853da 100644
--- a/main.tex
+++ b/main.tex
@@ -154,14 +154,16 @@ pipeline proceeds as follows:
     previously kept sample $\mathbf{s}'$.
 \end{enumerate}
 
-In addition to deduplication, instruction--response pairs whose
-response contains fewer than 15 tokens are discarded.
-Short replies exhibit high per-token variance in both
-conditional and unconditional loss (Appendix~\ref{sec:ifd_bias},
-Figure~\ref{fig:length_bias}), which would otherwise distort
-the IFD-based difficulty estimates used for downstream sample
-selection.  This length floor is applied per-field, analogous to
-the pretraining filter described above.
+A length filter is applied to SFT samples based on the IFD
+length-bias analysis in Appendix~\ref{sec:ifd_bias}
+(Figure~\ref{fig:length_bias}): instruction--response pairs
+whose response contains fewer than 15 tokens are discarded,
+because short replies exhibit both high per-token perplexity
+($L_{\text{uncond}} \approx 6\text{--}8$, PPL~$\approx 400\text{--}3000$)
+and wide variance in both $L_{\text{cond}}$ and $L_{\text{uncond}}$,
+which would distort downstream IFD-based difficulty estimates.
+The threshold is applied per-field, analogous to the pretraining
+filter described above.
 
 \subsection{DPO Data Generation}