refactor : grad_norm 指标简化,clip_grad_norm 移至 executor
- metrics 默认加入 grad_norm,移除 grad_std/max/min/mean/nan_num - grad_norm 默认返回总 L2 范数,per_param=True 返回各参数范数 - clip_grad_norm 从 callback 移至 BaseExecutor/FSDPExecutor - FSDPExecutor 覆盖为 model.clip_grad_norm_() 保证分布式正确 - ctx_get_grad_norm 改为读取 context.grad_norm
This commit is contained in:
@@ -150,8 +150,8 @@ def parse_args() -> argparse.Namespace:
|
||||
parser.add_argument(
|
||||
"--metrics",
|
||||
nargs="*",
|
||||
default=["loss", "lr"],
|
||||
help="Metrics to log (e.g. --metrics loss lr val_loss). Default: loss lr.",
|
||||
default=["loss", "lr", "grad_norm"],
|
||||
help="Metrics to log (e.g. --metrics loss lr val_loss). Default: loss lr grad_norm.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--log_dir",
|
||||
|
||||
Reference in New Issue
Block a user