refactor : neftune_alpha 在 Embedding 构造时传入，由模型配置链路负责

- BaseModelConfig 添加 neftune_alpha 字段 (默认 0.0) - Embedding.__init__ 接受 neftune_alpha 参数，不再外部 set - AutoRegressiveLM / EmbeddingEncoder 从 config 传入 neftune_alpha - train.py 将 CLI 参数注入 config 后再创建模型 - TrainContextBuilder 移除 neftune 设置（不再是其职责）
2026-06-19 14:23:27 +08:00
parent b1adc40cfb
commit 39985840c7
6 changed files with 13 additions and 5 deletions
@@ -59,7 +59,9 @@ class AutoRegressiveLM(AutoModel):
        self.rotary_embedding = RotaryEmbedding(
            rope_dim, config.max_len, rope_base, rope_scaling=config.rope_scaling
        )
-        self.embed_tokens = Embedding(config.vocab_size, config.dim)
+        self.embed_tokens = Embedding(
+            config.vocab_size, config.dim, neftune_alpha=config.neftune_alpha
+        )

        self.layers = nn.ModuleList(
            [DecoderBlock(config, layer_id) for layer_id in range(config.n_layers)]