 ViperEkura
|
995a015c23
|
fix: cache_h5 移除重复 --batch-size 参数，默认输出格式改为 bin
|
2026-08-19 18:01:06 +08:00 |
|
|
ViperEkura
|
cc451e5492
|
feat: add Ultra-FineWeb-L3 QA Synthetic pre-train export scripts (en/zh)
|
2026-07-31 10:03:49 +08:00 |
|
|
ViperEkura
|
e999629098
|
移除缓存脚本中的 data_files 和 max_chunks 限制参数
|
2026-07-26 10:57:30 +08:00 |
|
|
ViperEkura
|
fec37545e2
|
Add max_chunks=18 to dolma-30b export, remove max_chunks=5 from english-wiki, rename project to datapipline
|
2026-07-06 08:53:18 +08:00 |
|
 jiachun
|
efba7a009f
|
feat: add sft scripts for Magicoder-Evol-Instruct-110K and alpaca-gpt4-data
- sft_magicoder.py: export ise-uiuc/Magicoder-Evol-Instruct-110K
- sft_alpaca_gpt4.py: export llm-wizard/alpaca-gpt4-data with instruction+input
- remove deprecated SFT scripts
|
2026-06-01 15:50:13 +08:00 |
|
|
ViperEkura
|
f44ad6912e
|
fix: 修复特殊token 问题
|
2026-04-02 16:16:02 +08:00 |
|
|
ViperEkura
|
909f1b400c
|
refactor: 修改项目结构
|
2026-03-21 21:25:59 +08:00 |
|