fix: resolve audited training and inference bugs

- reject prompts that encode to zero tokens in add_task instead of admitting a task whose prefill can never run, and surface empty-id run_batch calls as prompt_empty errors
- deliver the STOP stream callback when cancelling a live task so clients observe termination instead of hanging until socket timeout
- strip the torch.compile _orig_mod. prefix at every unwrap_model site and when loading checkpoints so FSDP state dicts and saved weights no longer leak the wrapper name into downstream keys
- reject online_* train strategies with nprocs > 1 at config validation time, explaining the NCCL all-gather deadlock they would otherwise hit mid-run
- apply the frequency penalty before temperature scaling (OpenAI semantics) so the penalty survives temperature=0 instead of being annihilated by the 1e8 logit blowup, and exclude penalty pipelines from the greedy fast path
- return logprobs from the raw pre-strategy distribution so they match training-side policy logprobs for PPO/GRPO importance ratios
This commit is contained in:
2026-09-02 21:25:01 +08:00
parent 92e3cdf044
commit 88c06db096
8 changed files with 181 additions and 40 deletions
+4
View File
@@ -444,6 +444,10 @@ class InferenceScheduler:
tasks: List[Optional[Task]] = []
error_reasons: List[Optional[str]] = []
for ids in prompt_ids_list:
if not ids:
tasks.append(None)
error_reasons.append("prompt_empty")
continue
if len(ids) >= seq_cap:
tasks.append(None)
error_reasons.append("prompt_too_long")