ViperEkura/AstrAI
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

refactor : align config field names with Hugging Face

Browse Source 
- dim -> hidden_size, n_layers -> num_hidden_layers
- dim_ffn -> intermediate_size, n_heads -> num_attention_heads
- n_kv_heads -> num_key_value_heads, max_len -> max_position_embeddings
- norm_eps -> rms_norm_eps, tie_weight -> tie_word_embeddings
- update model, inference, training, scripts, tests, docs
This commit is contained in:
ViperEkura
2026-07-20 22:05:31 +08:00
parent d7ac66fb73
commit 0c86c89af4
23 changed files with 202 additions and 166 deletions
Download Patch File Download Diff File Expand all files Collapse all files
assets/docs/architecture.md
+15 -15
View File
@@ -28,17 +28,17 @@ classDiagram
class AutoRegressiveLMConfig {
+Optional[int] vocab_size
+Optional[int] dim
+Optional[int] n_layers
+Optional[float] norm_eps
+Optional[int] dim_ffn
+Optional[bool] tie_weight
+Optional[int] hidden_size
+Optional[int] num_hidden_layers
+Optional[float] rms_norm_eps
+Optional[int] intermediate_size
+Optional[bool] tie_word_embeddings
+Optional[dict] rope_scaling
+Optional[int] max_len
+Optional[int] max_position_embeddings
+Optional[float] rope_theta
+str attn_type
+Optional[int] n_heads
+Optional[int] n_kv_heads
+Optional[int] num_attention_heads
+Optional[int] num_key_value_heads
+Optional[bool] use_qk_norm
+Optional[bool] use_gated_attention
+Optional[int] kv_lora_rank
@@ -53,15 +53,15 @@ classDiagram
class EncoderConfig {
+Optional[int] vocab_size
+Optional[int] dim
+Optional[int] n_layers
+Optional[float] norm_eps
+Optional[int] dim_ffn
+Optional[int] max_len
+Optional[int] hidden_size
+Optional[int] num_hidden_layers
+Optional[float] rms_norm_eps
+Optional[int] intermediate_size
+Optional[int] max_position_embeddings
+Optional[float] rope_theta
+str attn_type
+Optional[int] n_heads
+Optional[int] n_kv_heads
+Optional[int] num_attention_heads
+Optional[int] num_key_value_heads
+Optional[bool] use_qk_norm
+str ffn_type
+Optional[dict] rope_scaling