diff --git a/main.tex b/main.tex index 2c2ba65..62e0120 100644 --- a/main.tex +++ b/main.tex @@ -381,10 +381,10 @@ for the base model. \begin{table}[H] \centering -\caption{Representative IFD samples (base model sorted by descending IFD).} +\caption{Representative IFD samples covering four patterns.} \label{tab:ifd_examples} \small -\begin{tabular}{@{}c c c c c c p{4.5cm}@{}} +\begin{tabular}{@{}c c c c c c c p{4.5cm}@{}} \toprule \textbf{Idx} & \textbf{Base IFD} & @@ -392,18 +392,16 @@ for the base model. \textbf{$L_{\text{cond}}^{\text{base}}$} & \textbf{$L_{\text{uncond}}^{\text{base}}$} & \textbf{$L_{\text{cond}}^{\text{ckpt}}$} & - \textbf{Instruction (truncated)} \\ + \textbf{$L_{\text{uncond}}^{\text{ckpt}}$} & + \textbf{Instruction} \\ \midrule -0 & 4.605 & 1.525 & 12.38 & 2.69 & 3.77 & Complete the following analogy \dots \\ -1 & 4.331 & 0.645 & 11.44 & 2.64 & 1.66 & Classify the following text \dots \\ -2 & 3.741 & 0.702 & 11.75 & 3.14 & 2.17 & Label the following news article \dots \\ -3 & 0.977 & 0.904 & 2.57 & 2.63 & 2.24 & Describe the role of a project manager \\ -4 & 0.977 & 0.915 & 2.19 & 2.25 & 1.98 & Select a historical figure \dots \\ -5 & 0.977 & 0.949 & 2.57 & 2.63 & 2.26 & Write the lyrics for an upbeat song \dots \\ -6 & 0.977 & 0.925 & 2.94 & 3.00 & 2.62 & Explain how neural networks \dots \\ -7 & 0.370 & 0.249 & 1.37 & 3.70 & 0.85 & Convert the given paragraph to a list \\ -8 & 0.338 & 0.197 & 0.98 & 2.91 & 0.55 & Insert a suitable greeting \dots \\ -9 & 0.307 & 0.062 & 0.70 & 2.29 & 0.15 & Remove third-person words \dots \\ +0 & 4.605 & 1.525 & 12.38 & 2.69 & 3.77 & 2.47 & Complete analogy: loud is to quiet as day is to \\ +1 & 3.741 & 0.702 & 11.75 & 3.14 & 2.17 & 3.09 & Label news article as ``Political'' or ``Entertainment'' \\ +2 & 1.044 & 0.089 & 3.50 & 3.35 & 0.28 & 3.10 & Find the capital of Spain \\ +3 & 1.056 & 0.147 & 4.09 & 3.88 & 0.60 & 4.07 & Edit sentence for correct grammar: ``I were just going to'' \\ +4 & 0.977 & 0.904 & 2.57 & 2.63 & 2.24 & 2.48 & Describe the role of a project manager \\ +5 & 0.370 & 0.249 & 1.37 & 3.70 & 0.85 & 3.42 & Convert the given paragraph to a list \\ +6 & 0.307 & 0.062 & 0.70 & 2.29 & 0.15 & 2.43 & Remove third-person words from sentence \\ \bottomrule \end{tabular} \end{table} @@ -431,28 +429,35 @@ Over $N=3000$ SFT samples: \subsection{Observed Patterns} -\paragraph{High-IFD samples (base IFD $> 3$, e.g.,~rows~0--2).} -These are tasks requiring task-intent comprehension: analogy completion, -text classification, article labeling. In the base model (15B pretraining), conditional +\paragraph{High-IFD samples (base IFD $> 3$, e.g.,~rows~0,~1).} +These are tasks requiring task-intent comprehension: analogy completion +and article labeling. In the base model (15B pretraining), conditional loss is extremely high ($L_{\text{cond}} \approx 12$), meaning the instruction still acts as noise. After 1K SFT steps, IFD drops sharply (e.g., $4.605 \rightarrow 1.525$), demonstrating that SFT teaches the model to interpret and follow abstract task descriptions. -\paragraph{Low-IFD samples (base IFD $< 0.4$, e.g.,~rows~7--9).} +\paragraph{Low-IFD samples (base IFD $< 0.4$, e.g.,~rows~5,~6).} These are formatting or extraction tasks: ``Convert paragraph to list,'' -``Remove third-person words,'' ``Insert a greeting.'' Unconditional +``Remove third-person words.'' Unconditional loss is much higher than conditional loss even in the base model, because the instruction naturally constrains the output space. The pattern persists after SFT but with lower absolute values. -\paragraph{Mid-range samples (base IFD $\approx 0.98$, e.g.,~rows~3--6).} -These cover factual Q\&A and generation tasks: ``Describe the role of -a project manager,'' ``Write lyrics for a song,'' ``Explain how neural -networks work.'' In the base model IFD $\approx 1$ (instruction has -little effect); after SFT IFD drops to $\approx 0.9$, driven by -a clear reduction in conditional loss. +\paragraph{Mid-range with large drop (e.g.,~rows~2,~3).} +These are factual QA or grammar correction tasks. Base IFD is +$\approx 1.05$ (instruction has little effect), but after SFT +IFD drops to $\approx 0.1$ as the model learns the precise answer +(e.g., ``Madrid'' for ``capital of Spain''), making conditional loss +near-zero while unconditional loss remains high. + +\paragraph{Mid-range with small drop (e.g.,~row~4).} +These are open-ended generation tasks (``Describe the role of a +project manager''). Base IFD $\approx 0.98$; after SFT it drops +only modestly to $\approx 0.9$, since both conditional and +unconditional losses decrease proportionally without a memorized +target. \paragraph{Cross-model correlation.} The moderate Pearson correlation ($r = 0.38$) suggests that while