docs: align CUDA kernel and RoPE docs with code
- Fix rotary docs to describe cos/sin freqs_cis table, not complex buffer - Replace attn_prefill with attn_paged_prefill for the CudaBackend path - Register attn_paged_prefill in kernel overview, layout, and module list - Add qo_indptr and InferenceWorkspace to architecture class diagram - Add FrequencyPenaltyStrategy to sampling design patterns
This commit is contained in:
@@ -41,10 +41,12 @@ RoPE embeds position into Q/K vectors via complex rotation:
|
||||
|
||||
$$ q_i = R_i W_q x_i, \quad k_j = R_j W_k x_j, \quad q_i^T k_j = x_i^T W_q^T R_{i-j} W_k x_j $$
|
||||
|
||||
`RotaryEmbedding` pre-computes a complex `freqs_cis` buffer. `forward()` returns
|
||||
a tensor indexed by `position_ids`. `apply_rotary_emb` applies the rotation:
|
||||
during training it uses torch complex multiply (autograd-compatible); during
|
||||
inference it auto-dispatches to a fused CUDA kernel when available.
|
||||
`RotaryEmbedding` pre-computes a cos/sin table `freqs_cis` of shape
|
||||
`[max_len, dim/2, 2]` (f32 — `[cos, sin]` pairs). `forward()` returns
|
||||
a `[batch, seq_len, dim/2, 2]` slice indexed by `position_ids`.
|
||||
`apply_rotary_emb` applies the rotation: during training it uses torch
|
||||
complex multiply (autograd-compatible); during inference it auto-dispatches
|
||||
to a fused CUDA kernel when available.
|
||||
|
||||
## Training Loop
|
||||
|
||||
|
||||
Reference in New Issue
Block a user