perf: 消除 PagedCache.write 中的 position_ids GPU 同步,解码提速 15%
- CacheView.write 用 total_len - k.size(1) 推导 start_pos,替代 position_ids[0,0].item() - 移除 GQA/MLA/DecoderBlock 中不再使用的 position_ids 参数 - PagedCache.write 参数 position_ids:Tensor → start_pos:int
This commit is contained in:
@@ -132,7 +132,7 @@ class Transformer(AutoModel):
|
||||
attn_mask = process_attention_mask(x, position_ids, input_mask, is_causal=True)
|
||||
|
||||
for layer in self.layers:
|
||||
x = layer(x, rotary_emb, attn_mask, paged_cache, position_ids)
|
||||
x = layer(x, rotary_emb, attn_mask, paged_cache)
|
||||
|
||||
hidden_states = self.norm(x)
|
||||
logits = self.lm_head(hidden_states)
|
||||
|
||||
Reference in New Issue
Block a user