This website requires JavaScript.
Explore
Help
Register
Sign In
ViperEkura
/
DataPipeline
Watch
1
Star
0
Fork
0
Code
Issues
Pull Requests
Packages
Projects
Releases
Wiki
Activity
Files
d67f686f10ebdf9f8245bce303beea2ac274d6b4
DataPipeline
/
scripts
T
History
ViperEkura
d67f686f10
merge remote main
2026-08-05 12:47:31 +08:00
..
pre_train
feat: add Ultra-FineWeb-L3 QA Synthetic pre-train export scripts (en/zh)
2026-07-31 10:03:49 +08:00
reforce_learning
reafactor: 重构项目
2026-03-30 20:58:51 +08:00
supervised_finetuning
feat: add Alpaca-GPT4-ZH SFT export script
2026-07-29 21:23:20 +08:00
cache_h5.py
merge remote main
2026-08-05 12:47:31 +08:00
dedup_pretrain.py
feat: MinHash+LSH 去重 + Strategy/Factory 存储后端
2026-07-04 14:47:38 +08:00
filter_short.py
feat: add SFT process_batch for parallel tokenization + short QA filter script
2026-07-29 22:01:09 +08:00