refactor : align config field names with Hugging Face
- dim -> hidden_size, n_layers -> num_hidden_layers - dim_ffn -> intermediate_size, n_heads -> num_attention_heads - n_kv_heads -> num_key_value_heads, max_len -> max_position_embeddings - norm_eps -> rms_norm_eps, tie_weight -> tie_word_embeddings - update model, inference, training, scripts, tests, docs
This commit is contained in:
@@ -240,7 +240,7 @@ class TrainContextBuilder:
|
||||
|
||||
group_size = strategy_kwargs.get("group_size", 1)
|
||||
rollout_batch_size = group_size * max(1, cfg.batch_per_device)
|
||||
max_seq_len = getattr(context.model.config, "max_len", None)
|
||||
max_seq_len = getattr(context.model.config, "max_position_embeddings", None)
|
||||
|
||||
scheduler = InferenceScheduler(
|
||||
model=context.model,
|
||||
|
||||
Reference in New Issue
Block a user