feat: load HuggingFace checkpoints via key/config conversion
- Add astrai.serialization.hf_adapter mapping LLaMA-style HF keys to AstrAI names (input_layernorm, gate_proj, MoE experts/shared_experts) with config aliases for dense and MoE (Mixtral/DeepSeek-V3) layouts; reject biased projections, mismatched head_dim and MLA - Give AutoModel.from_pretrained weights_format=auto|astrai|hf with auto-detection; read sharded safetensors via model.safetensors.index.json - Adapt preloaded weights/config in train_context and benchmark CLI
This commit is contained in:
@@ -58,6 +58,14 @@ The model directory contains:
|
||||
- `model.safetensors` — model weights
|
||||
- `tokenizer.json` + `tokenizer_config.json` — tokenizer files (including chat template)
|
||||
|
||||
External HuggingFace checkpoints of the LLaMA layout (e.g. `meta-llama/...`,
|
||||
`mistralai/...`, `Qwen/Qwen2-...`) can be loaded directly: `AutoModel.from_pretrained`
|
||||
auto-detects HF `model_type` / key names (`input_layernorm`, `gate_proj`, MoE
|
||||
`experts.<j>` ...) and converts config and weights in place. Dense and MoE
|
||||
(Mixtral / DeepSeek-V3 layout) FFNs are supported; MLA attention
|
||||
(DeepSeek-V2/V3) and biased projections (`attention_bias`) are not. Pass
|
||||
`weights_format="astrai"` to skip conversion, or `"hf"` to force it.
|
||||
|
||||
## 3. Run Inference
|
||||
|
||||
### Interactive Chat (Simplest)
|
||||
|
||||
Reference in New Issue
Block a user