feat: load HuggingFace checkpoints via key/config conversion

- Add astrai.serialization.hf_adapter mapping LLaMA-style HF keys to AstrAI names (input_layernorm, gate_proj, MoE experts/shared_experts) with config aliases for dense and MoE (Mixtral/DeepSeek-V3) layouts; reject biased projections, mismatched head_dim and MLA
- Give AutoModel.from_pretrained weights_format=auto|astrai|hf with auto-detection; read sharded safetensors via model.safetensors.index.json
- Adapt preloaded weights/config in train_context and benchmark CLI
This commit is contained in:
2026-08-20 11:34:59 +08:00
parent 84753d3e08
commit 7d27f3e078
8 changed files with 645 additions and 8 deletions
+8
View File
@@ -58,6 +58,14 @@ The model directory contains:
- `model.safetensors` — model weights
- `tokenizer.json` + `tokenizer_config.json` — tokenizer files (including chat template)
External HuggingFace checkpoints of the LLaMA layout (e.g. `meta-llama/...`,
`mistralai/...`, `Qwen/Qwen2-...`) can be loaded directly: `AutoModel.from_pretrained`
auto-detects HF `model_type` / key names (`input_layernorm`, `gate_proj`, MoE
`experts.<j>` ...) and converts config and weights in place. Dense and MoE
(Mixtral / DeepSeek-V3 layout) FFNs are supported; MLA attention
(DeepSeek-V2/V3) and biased projections (`attention_bias`) are not. Pass
`weights_format="astrai"` to skip conversion, or `"hf"` to force it.
## 3. Run Inference
### Interactive Chat (Simplest)