fix: 修正AdamW超参数默认值与文档示例
- 交换adamw_beta1/adamw_beta2默认值:beta1=0.95, beta2=0.99 - label_smoothing默认值改为0.05 - 文档示例统一更新:train_type=pt, weight_decay=0.01 - 移除文档中过时的strategy default标注
This commit is contained in:
@@ -82,7 +82,7 @@ export CUDA_VISIBLE_DEVICES=0,1,2,3
|
||||
|
||||
nohup python scripts/tools/train.py \
|
||||
--nprocs=4 \
|
||||
--train_type=sft \
|
||||
--train_type=pt \
|
||||
--data_root_path=/path/to/dataset \
|
||||
--param_path=/path/to/model \
|
||||
--batch_per_device=4 \
|
||||
@@ -90,14 +90,14 @@ nohup python scripts/tools/train.py \
|
||||
--warmup_ratio=0.05 \
|
||||
--max_lr=1e-4 \
|
||||
--max_grad_norm=1.0 \
|
||||
--adamw_beta1=0.99 \
|
||||
--adamw_beta2=0.95 \
|
||||
--adamw_weight_decay=1e-5 \
|
||||
--adamw_beta1=0.95 \
|
||||
--adamw_beta2=0.99 \
|
||||
--adamw_weight_decay=0.01 \
|
||||
--window_size=2048 \
|
||||
--ckpt_interval=10000 \
|
||||
--ckpt_dir=./checkpoint \
|
||||
--random_seed=3407 \
|
||||
--label_smoothing=0.1 \
|
||||
--label_smoothing=0.05 \
|
||||
> out.log 2> err.log &
|
||||
```
|
||||
|
||||
|
||||
Reference in New Issue
Block a user