fix : SFT pipeline position_ids default & doc boundary preservation

- change position_ids_mode default from "none" to "doc_reset" so SFT preprocessing always generates position_ids (was causing dataset load KeyError) - generate per-doc position_ids before packing (doc_reset mode), preserving document boundaries for BFD packing (cross-doc attention leak fix) - change _align_bucket padding from [1] to [0] to avoid accidentally training on loss_mask padding
2026-07-04 15:59:11 +08:00
parent 8999ca89b8
commit 4e508afa2d
3 changed files with 13 additions and 6 deletions
@@ -96,7 +96,7 @@ class OutputConfig(BaseConfig):
    storage_format: str = "bin"
    max_tokens_per_shard: int = 100_000_000
    dtype: Dict[str, str] = field(default_factory=dict)
-    position_ids_mode: str = "none"
+    position_ids_mode: str = "doc_reset"


@dataclass