- get_rotary_emb() 返回复数张量替代 Tuple[cos, sin] - RotaryEmbedding 存储单一 freqs_cis buffer 替代分离的 cos_cached/sin_cached - forward 中 view_as_complex 重建复数 |
||
|---|---|---|
| .. | ||
| config | ||
| dataset | ||
| inference | ||
| model | ||
| parallel | ||
| tokenize | ||
| trainer | ||
| __init__.py | ||
| factory.py | ||
| serialization.py | ||
- get_rotary_emb() 返回复数张量替代 Tuple[cos, sin] - RotaryEmbedding 存储单一 freqs_cis buffer 替代分离的 cos_cached/sin_cached - forward 中 view_as_complex 重建复数 |
||
|---|---|---|
| .. | ||
| config | ||
| dataset | ||
| inference | ||
| model | ||
| parallel | ||
| tokenize | ||
| trainer | ||
| __init__.py | ||
| factory.py | ||
| serialization.py | ||