refactor : 将 config 对象直接传给 DecoderBlock,替代 16 个独立参数
- DecoderBlock.__init__ 改为 (config, layer_id),内部用 asdict 展开字段给 AttnFactory/FFNFactory,factory 按 __init__ 签名自动过滤 - EncoderConfig 补充 attn_type 和 ffn_type 字段 - 314 个测试全部通过
This commit is contained in:
@@ -70,10 +70,12 @@ class EncoderConfig(BaseModelConfig):
|
||||
rope_theta: Optional[float] = None
|
||||
rope_scaling: Optional[dict] = None
|
||||
|
||||
attn_type: str = "gqa"
|
||||
n_heads: Optional[int] = None
|
||||
n_kv_heads: Optional[int] = None
|
||||
use_qk_norm: Optional[bool] = None
|
||||
use_gated_attention: Optional[bool] = None
|
||||
|
||||
ffn_type: str = "mlp"
|
||||
pooling_type: Optional[str] = None
|
||||
normalize_embeddings: Optional[bool] = None
|
||||
|
||||
Reference in New Issue
Block a user