refactor : 将 config 对象直接传给 DecoderBlock,替代 16 个独立参数

- DecoderBlock.__init__ 改为 (config, layer_id),内部用 asdict
  展开字段给 AttnFactory/FFNFactory,factory 按 __init__ 签名自动过滤
- EncoderConfig 补充 attn_type 和 ffn_type 字段
- 314 个测试全部通过
This commit is contained in:
2026-06-19 14:15:33 +08:00
parent 7348bac6ab
commit b1adc40cfb
4 changed files with 11 additions and 63 deletions
+2
View File
@@ -70,10 +70,12 @@ class EncoderConfig(BaseModelConfig):
rope_theta: Optional[float] = None
rope_scaling: Optional[dict] = None
attn_type: str = "gqa"
n_heads: Optional[int] = None
n_kv_heads: Optional[int] = None
use_qk_norm: Optional[bool] = None
use_gated_attention: Optional[bool] = None
ffn_type: str = "mlp"
pooling_type: Optional[str] = None
normalize_embeddings: Optional[bool] = None