feat: load HuggingFace checkpoints via key/config conversion

- Add astrai.serialization.hf_adapter mapping LLaMA-style HF keys to AstrAI names (input_layernorm, gate_proj, MoE experts/shared_experts) with config aliases for dense and MoE (Mixtral/DeepSeek-V3) layouts; reject biased projections, mismatched head_dim and MLA
- Give AutoModel.from_pretrained weights_format=auto|astrai|hf with auto-detection; read sharded safetensors via model.safetensors.index.json
- Adapt preloaded weights/config in train_context and benchmark CLI
This commit is contained in:
2026-08-20 11:34:59 +08:00
parent 84753d3e08
commit 7d27f3e078
8 changed files with 645 additions and 8 deletions
+12
View File
@@ -22,9 +22,21 @@ from astrai.serialization.dataset import (
load_bin_offsets,
save_bin,
)
from astrai.serialization.hf_adapter import (
HF_MODEL_TYPES,
adapt_config,
convert_hf_config,
convert_hf_weights,
looks_like_hf_state_dict,
)
__all__ = [
"Checkpoint",
"HF_MODEL_TYPES",
"adapt_config",
"convert_hf_config",
"convert_hf_weights",
"looks_like_hf_state_dict",
"load_json",
"load_model_config",
"load_model_weights",