- gqa_decode_attn: rewrite to per-KV-head, K in smem - gqa_prefill_attn: new kernel for Q_len > 1 with GQA |
||
|---|---|---|
| .. | ||
| config | ||
| dataset | ||
| extension | ||
| inference | ||
| model | ||
| parallel | ||
| preprocessing | ||
| serialization | ||
| tokenize | ||
| trainer | ||
| __init__.py | ||
| factory.py | ||
| protocols.py | ||
- gqa_decode_attn: rewrite to per-KV-head, K in smem - gqa_prefill_attn: new kernel for Q_len > 1 with GQA |
||
|---|---|---|
| .. | ||
| config | ||
| dataset | ||
| extension | ||
| inference | ||
| model | ||
| parallel | ||
| preprocessing | ||
| serialization | ||
| tokenize | ||
| trainer | ||
| __init__.py | ||
| factory.py | ||
| protocols.py | ||