fix: process_attention_mask 中 expand 后的 inplace 写导致 alias 报错
- pad.view.expand 产生的视图多元素指向同一内存,attend &= 写入报错 - 改为 .expand().clone() 独立内存后再 inplace
This commit is contained in:
parent
7e26d848ab
commit
466c2e1efd
|
|
@ -39,7 +39,7 @@ def process_attention_mask(
|
|||
else:
|
||||
pad = input_mask[:, :T].to(device=device, dtype=torch.bool)
|
||||
|
||||
attend = pad.view(B, 1, T).expand(B, S, T)
|
||||
attend = pad.view(B, 1, T).expand(B, S, T).clone()
|
||||
if is_causal:
|
||||
attend &= position_ids.unsqueeze(-1) >= torch.arange(T, device=device)
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue