AstrAI/scripts/eval
ViperEkura db9b39b084 fix: resolve IFD token-set asymmetry and support single-token answers
- Sentinel-anchored unconditional pass: both branches now predict the same N response tokens
- Single-token responses (rl=1) fully supported
- ctx_len tracked per sample; skip_reason replaces silent None
- --per_token flag for per-token IFD breakdown
2026-07-05 17:48:26 +08:00
..
analyze_weights.py feat: SVD 有效秩/权重统计分析脚本 2026-06-29 21:39:22 +08:00
evaluate_humaneval.py fix: reliable test timeout, separate generate/test phases, dynamic pass@k 2026-07-05 08:47:30 +08:00
evaluate_ifd.py fix: resolve IFD token-set asymmetry and support single-token answers 2026-07-05 17:48:26 +08:00
evaluate_ifeval.py refactor : metric_logger 改用事件类型 (type=step/validation/epoch) 2026-06-25 17:18:20 +08:00
evaluate_mmlu.py feat: 新增 IFEval 指令遵循评测 2026-06-16 21:57:34 +08:00
evaluate_ppl.py fix: 规范 generate.py 命令行接口 2026-06-19 14:03:02 +08:00
evaluate_rouge.py feat : add ROUGE evaluation script (manual impl, no deps) 2026-07-05 01:15:01 +08:00