refactor : align config field names with Hugging Face
- dim -> hidden_size, n_layers -> num_hidden_layers - dim_ffn -> intermediate_size, n_heads -> num_attention_heads - n_kv_heads -> num_key_value_heads, max_len -> max_position_embeddings - norm_eps -> rms_norm_eps, tie_weight -> tie_word_embeddings - update model, inference, training, scripts, tests, docs
This commit is contained in:
+15
-15
@@ -28,17 +28,17 @@ classDiagram
|
||||
|
||||
class AutoRegressiveLMConfig {
|
||||
+Optional[int] vocab_size
|
||||
+Optional[int] dim
|
||||
+Optional[int] n_layers
|
||||
+Optional[float] norm_eps
|
||||
+Optional[int] dim_ffn
|
||||
+Optional[bool] tie_weight
|
||||
+Optional[int] hidden_size
|
||||
+Optional[int] num_hidden_layers
|
||||
+Optional[float] rms_norm_eps
|
||||
+Optional[int] intermediate_size
|
||||
+Optional[bool] tie_word_embeddings
|
||||
+Optional[dict] rope_scaling
|
||||
+Optional[int] max_len
|
||||
+Optional[int] max_position_embeddings
|
||||
+Optional[float] rope_theta
|
||||
+str attn_type
|
||||
+Optional[int] n_heads
|
||||
+Optional[int] n_kv_heads
|
||||
+Optional[int] num_attention_heads
|
||||
+Optional[int] num_key_value_heads
|
||||
+Optional[bool] use_qk_norm
|
||||
+Optional[bool] use_gated_attention
|
||||
+Optional[int] kv_lora_rank
|
||||
@@ -53,15 +53,15 @@ classDiagram
|
||||
|
||||
class EncoderConfig {
|
||||
+Optional[int] vocab_size
|
||||
+Optional[int] dim
|
||||
+Optional[int] n_layers
|
||||
+Optional[float] norm_eps
|
||||
+Optional[int] dim_ffn
|
||||
+Optional[int] max_len
|
||||
+Optional[int] hidden_size
|
||||
+Optional[int] num_hidden_layers
|
||||
+Optional[float] rms_norm_eps
|
||||
+Optional[int] intermediate_size
|
||||
+Optional[int] max_position_embeddings
|
||||
+Optional[float] rope_theta
|
||||
+str attn_type
|
||||
+Optional[int] n_heads
|
||||
+Optional[int] n_kv_heads
|
||||
+Optional[int] num_attention_heads
|
||||
+Optional[int] num_key_value_heads
|
||||
+Optional[bool] use_qk_norm
|
||||
+str ffn_type
|
||||
+Optional[dict] rope_scaling
|
||||
|
||||
@@ -85,7 +85,7 @@ All backends normalise tensors into `Store._data[Dict[str, List[Tensor]]]` + `St
|
||||
```
|
||||
DatasetFactory.load(train_type, load_path, window_size, stride=None,
|
||||
storage_type=None, tokenizer_path=None,
|
||||
max_len=2048, store=None)
|
||||
max_position_embeddings=2048, store=None)
|
||||
→ BaseDataset.load(load_path, storage_type=None)
|
||||
→ detect_format(load_path)
|
||||
→ StoreFactory.create(storage_type)
|
||||
|
||||
@@ -32,7 +32,7 @@ ContiguousCache (simple contiguous per-slot cache)
|
||||
├── ContiguousCacheView bundles k/v tensors + slot indices for attention layers
|
||||
```
|
||||
|
||||
Created by default when no cache is passed to `InferenceScheduler`. Each task occupies a fixed slot of `[max_seq_len, n_kv_heads, head_dim]`. Simple and efficient for small-to-medium batch sizes.
|
||||
Created by default when no cache is passed to `InferenceScheduler`. Each task occupies a fixed slot of `[max_seq_len, num_key_value_heads, head_dim]`. Simple and efficient for small-to-medium batch sizes.
|
||||
|
||||
### PageCache (paged with prefix sharing)
|
||||
|
||||
@@ -42,7 +42,7 @@ PageCache (paged KV cache with prefix sharing, alternative)
|
||||
│ ├── Allocator bitmask-based page allocator + ref-count + LRU
|
||||
│ └── PrefixCache hash-based prefix matching (page_hash via polynomial hash)
|
||||
├── TaskTable maps task_id → page_table + cached token count
|
||||
├── Storage k_cache / v_cache tensors (n_layers × n_pages × page_size × n_kv_heads × head_dim)
|
||||
├── Storage k_cache / v_cache tensors (num_hidden_layers × n_pages × page_size × num_key_value_heads × head_dim)
|
||||
└── PageCacheView bundles Storage + page_table + total_len for attention layers
|
||||
```
|
||||
|
||||
|
||||
@@ -44,7 +44,7 @@ Combined optimizer: matrix parameters via **Muon**, non-matrix via **AdamW** (`f
|
||||
|
||||
| Parameter | Description | Default |
|
||||
|-----------|-------------|---------|
|
||||
| `--window_size` | Max input sequence length | model config `max_len` |
|
||||
| `--window_size` | Max input sequence length | model config `max_position_embeddings` |
|
||||
| `--stride` | Stride for sliding window over sequences | None |
|
||||
| `--random_seed` | Random seed for reproducibility | 3407 |
|
||||
| `--num_workers` | DataLoader worker processes | 4 |
|
||||
@@ -186,7 +186,7 @@ See [Inference Guide](inference.md) for HTTP API documentation.
|
||||
| `--top_k` | int | `30` | Top-k filtering |
|
||||
| `--top_p` | float | `0.95` | Nucleus sampling threshold |
|
||||
| `--batch_size` | int | `1` | Batch size for generation |
|
||||
| `--max_tokens` | int | model config `max_len` | Maximum tokens to generate |
|
||||
| `--max_tokens` | int | model config `max_position_embeddings` | Maximum tokens to generate |
|
||||
|
||||
Usage:
|
||||
```bash
|
||||
|
||||
Reference in New Issue
Block a user