refactor: map instruction/input/output to chat roles

- RolloutGenerator._instruction_to_messages builds system/user/assistant list (instruction->system, input->user, output->assistant), replacing single-user-turn concatenation
- Remove _iter_samples helper; _prepare_prompts zips parallel list-of-strings fields directly per the collate_fn contract
- Tests adopt a system-aware chat template and pin the three-field role mapping
- Drop unused imports caught by ruff F401 (torch.Tensor in scheduler.py, iter_raw_records in pipeline.py, Tuple in evaluate_rouge.py)
This commit is contained in:
2026-07-20 13:55:25 +08:00
parent e8ff7f5321
commit 06eeeead79
6 changed files with 239 additions and 84 deletions
-1
View File
@@ -23,7 +23,6 @@ import tqdm
from astrai.config.preprocess_config import PipelineConfig
from astrai.preprocessing.core import (
build_preprocessing_components,
iter_raw_records,
primary_ids,
)
from astrai.preprocessing.packing import PackingStrategyFactory