refactor: 改用递归子模块 init 替代统一 normal_(0.006)

- Embedding.reset_parameters: normal_(std=0.02)
- Linear.reset_parameters: kaiming_uniform_ + uniform_ bias
- Transformer._init_weights 通过 apply 递归调用子模块 reset_parameters
- 移除全局 normal_(0.006) 覆盖,各模块使用更合适的分布
This commit is contained in:
2026-05-17 10:44:18 +08:00
parent ad9f4d9cf6
commit 1d54491809
3 changed files with 14 additions and 5 deletions
+3
View File
@@ -9,5 +9,8 @@ class Embedding(nn.Module):
super().__init__()
self.weight = nn.Parameter(torch.empty((vocab_size, embedding_dim)))
def reset_parameters(self):
nn.init.normal_(self.weight, mean=0.0, std=0.02)
def forward(self, x: Tensor) -> Tensor:
return F.embedding(x, self.weight)