ViperEkura fec37545e2 Add max_chunks=18 to dolma-30b export, remove max_chunks=5 from english-wiki, rename project to datapipline

2026-07-06 08:53:18 +08:00

..

pre_train

Add max_chunks=18 to dolma-30b export, remove max_chunks=5 from english-wiki, rename project to datapipline

2026-07-06 08:53:18 +08:00

reforce_learning

reafactor: 重构项目

2026-03-30 20:58:51 +08:00

supervised_finetuning

refactor: SFT 统一 messages 格式 + ChatML 纯 jinja 渲染

2026-07-04 14:32:35 +08:00

cache_h5.py

feat: MinHash+LSH 去重 + Strategy/Factory 存储后端

2026-07-04 14:47:38 +08:00

dedup_pretrain.py

feat: MinHash+LSH 去重 + Strategy/Factory 存储后端

2026-07-04 14:47:38 +08:00