ViperEkura

ViperEkura pushed to master at ViperEkura/AstrAI-paper

2026-07-05 00:17:27 +08:00

4a7bf7ac3f Add GQA/SwiGLU/RoPE formula blocks, Alpaca-GPT4 citation, and improve IFD-Loss Ratio interpretation

ViperEkura pushed to master at ViperEkura/AstrAI-paper

2026-07-04 23:46:39 +08:00

5c915514b3 Add correlation analysis interpretation to IFD bias section

ViperEkura pushed to master at ViperEkura/AstrAI-paper

2026-07-04 23:33:30 +08:00

bc8d4066c1 Add IFD analysis: density/length-bias figures and appendix

ViperEkura pushed to master at ViperEkura/AstrAI-paper

2026-07-04 22:39:27 +08:00

9534316bac Fix duplicate paragraph and misleading table caption

ViperEkura pushed to master at ViperEkura/AstrAI-paper

2026-07-04 22:31:56 +08:00

b40fcc86c7 Refine abstract, switch to Times font, remove date, fix overfull hboxes with microtype

ViperEkura pushed to master at ViperEkura/AstrAI-paper

2026-07-04 22:25:35 +08:00

dd412b3c9c Add IFD analysis, appendix, and improve abstract

ViperEkura pushed to main at ViperEkura/AstrAI

2026-07-04 22:13:07 +08:00

c7158418dd perf: add BFD bin-packing and custom attention mask to IFD batch scoring

4d3c9341c1 refactor: rewrite IFD evaluation with clean three-layer architecture

4e508afa2d fix : SFT pipeline position_ids default & doc boundary preservation

Compare 3 commits »

ViperEkura pushed to main at ViperEkura/AstrAI

2026-07-04 15:46:56 +08:00

8999ca89b8 feat: add JSONL dataset store with on-the-fly tokenization

1adca39cd8 fix: handle long sequences and optimize IFD computation

204873fa2f fix: handle long sequences and optimize IFD computation

a5c1de6b1b feat: add model_path temperature top_p top_k max_tokens system_prompt args to stream_chat

27524ad085 fix: reset sampler iter at epoch end so progress bar shows total after first epoch

Compare 15 commits »

ViperEkura pushed to master at ViperEkura/AstrAI-paper

2026-06-28 16:57:49 +08:00

73cb02f0aa fix training hyperparams to match pretraining script: lr 1.5e-4, wd 0.1, warmup 0.02

ViperEkura pushed to master at ViperEkura/AstrAI-paper

2026-06-28 16:52:20 +08:00

adc284920a add Alembic SFT data cleaning section with MinHash algorithm and fix bibliography order

ViperEkura pushed to master at ViperEkura/AstrAI-paper

2026-06-28 15:50:07 +08:00

13089b002a fix training config: grad_accum 32, remove unused validation & activation ckpt

ViperEkura pushed to master at ViperEkura/AstrAI-paper

2026-06-28 14:36:18 +08:00

d3fbe0ecc6 refactor: numerical stability analysis with residual scaling comparison

ViperEkura pushed to main at ViperEkura/AstrAI

2026-06-27 16:12:33 +08:00

6715461a36 chore : 升级 torch 2.11.0+cu128，移除自定义 Muon，修复 gloo device_id

ViperEkura pushed to main at ViperEkura/AstrAI

2026-06-25 17:22:45 +08:00

b4587c5d08 refactor : metric_logger 改用事件类型 (type=step/validation/epoch)

88ec63121d feat : GPT-2 residual scaling weight init

01d2da2893 feat : 训练支持 --schedule_type 及对应调度器参数

25d4ea3f91 refactor : 压缩测试代码，消除重复

39985840c7 refactor : neftune_alpha 在 Embedding 构造时传入，由模型配置链路负责

Compare 7 commits »

ViperEkura pushed to master at ViperEkura/AstrAI-paper

2026-06-25 16:53:35 +08:00

3db1096c78 fix: add caption package, fix overfull hbox, correct RoFormer author names

ViperEkura pushed to master at ViperEkura/AstrAI-paper

2026-06-25 15:51:14 +08:00

6f9278b35b Initial commit: BF16 weight lock-in paper for 1.2B Transformer training with AstrAI

ViperEkura created branch master in ViperEkura/AstrAI-paper

2026-06-25 15:51:14 +08:00

ViperEkura created repository ViperEkura/AstrAI-paper

2026-06-25 15:49:38 +08:00

ViperEkura pushed to main at ViperEkura/postgraduate-prep

2026-06-20 18:11:54 +08:00

2586583e0e feat: 新增重积分章节（笔记+错题）

ViperEkura pushed to main at ViperEkura/AstrAI

2026-06-19 13:54:20 +08:00

8ab7564d02 docs: 重构 README 结构，全文档添加目录导航

d096b6e29e docs: 修复文档中过时的字段、签名和缺失的类

d88a41f8f1 fix: 修复预处理流水线 4 个致命问题

376e9eba80 feat: IFEval 使用 chat template 格式化 prompt，添加 model.eval()

a62c2e11a2 feat: IFD 默认使用 chat template，支持裸文本模式

Compare 8 commits »