docs: scope trainer environment variables per job
- Add a Per-Job Environment section explaining that runtime.environment reaches only the GPUs declared in the same job YAML, with one-YAML-per-GPU-group examples for local, cross-PCIe workaround, and NVSwitch NVLink tuning setups - Replace the NCCL workaround pair in the runtime schema example with ASTR_LOG_LEVEL and ASTR_BACKEND and document value semantics (str() rendering, null exports empty, no host-shell passthrough) - Comment out the blanket NCCL exports in the get-started multi-GPU example so they are opt-in per docs/guides/distributed.md - Add a hard rule against copying NCCL workarounds into every training config
This commit is contained in:
+3
-2
@@ -190,8 +190,9 @@ python scripts/tools/train.py \
|
||||
|
||||
```bash
|
||||
export CUDA_VISIBLE_DEVICES=0,1,2,3
|
||||
export NCCL_P2P_DISABLE=1
|
||||
export NCCL_NET_GDR_LEVEL=0
|
||||
# Only if this host's NCCL transport is broken; see docs/guides/distributed.md:
|
||||
# export NCCL_P2P_DISABLE=1
|
||||
# export NCCL_NET_GDR_LEVEL=0
|
||||
|
||||
python scripts/tools/train.py \
|
||||
--train_type=seq \
|
||||
|
||||
Reference in New Issue
Block a user