AstrAI/astrai/preprocessing
ViperEkura 841a582b28 refactor: split mask builder by single/multi output
- Extract SingleOutputMaskBuilder for SFT and pretrain configs
- Extract MultiOutputMaskBuilder for DPO and GRPO configs
- Keep SectionedMaskBuilder as backward-compatible facade
- Register "single" and "multi" names in MaskBuilderFactory
- Add parity and rejection tests for concrete builders
2026-07-08 21:18:34 +08:00
..
__init__.py refactor: split mask builder by single/multi output 2026-07-08 21:18:34 +08:00
builder.py refactor: split mask builder by single/multi output 2026-07-08 21:18:34 +08:00
packing.py fix: 修复预处理流水线 4 个致命问题 2026-06-18 17:38:01 +08:00
pipeline.py fix : SFT pipeline position_ids default & doc boundary preservation 2026-07-04 15:59:11 +08:00
position_id.py fix : 修复策略相关文件的类型注解与抽象方法体 2026-06-14 16:20:10 +08:00
writer.py feat: add JSONL dataset store with on-the-fly tokenization 2026-07-04 15:42:33 +08:00