Gaolingx 6d98bb4f9f 20260801-moe model impl ...

need to add aux loss for load balancing

2026-08-01 22:48:58 +08:00

..

__init__.py

20260801-moe model impl

2026-08-01 22:48:58 +08:00

attention.py

perf: add fused CUDA rotary embedding kernel

2026-07-31 15:27:31 +08:00

decoder_block.py

20260801-moe model impl

2026-08-01 22:48:58 +08:00

embedding.py

refactor : neftune_alpha 在 Embedding 构造时传入，由模型配置链路负责

2026-06-19 14:23:27 +08:00

linear.py

feat : GPT-2 residual scaling weight init

2026-06-25 15:08:31 +08:00

lora.py

refactor: migrate config system to Pydantic dataclasses

2026-07-30 08:25:32 +08:00

mlp.py

20260801-moe model impl

2026-08-01 22:48:58 +08:00

norm.py

refactor: 拆分 module.py 为 components 子包

2026-05-15 20:08:36 +08:00

rope.py

refactor: unify rotary embedding interface and update docs

2026-07-31 16:52:25 +08:00