ViperEkura
115192c67c
refactor: remove H5 storage backend in favor of mmap bin
...
- Remove H5Store, H5Writer, save_h5/load_h5 and h5py dependency
- MmapStore (bin) is the sole pre-tokenized storage backend
- Move setup_logging after imports to fix E402 in __init__.py
- Clean up unused imports across test files
- Move inline test imports to file top
2026-07-29 12:50:27 +08:00
ViperEkura
5ba21f4eb3
refactor: eliminate test duplication via shared helpers
...
- Add tests/helpers.py with shared config, dataset, tokenizer, executor, and assertion helpers
- Replace 15 copies of device one-liner with session-scoped fixture
- Collapse 5 near-identical Dataset subclasses into RandomTokenDataset
- Remove duplicate _make_config/_make_model/_make_frozen and FakeTokenizer/FakeExecutor definitions
- Make test_callbacks and test_early_stopping use existing train_config_factory
- Replace 6 duplicate meta.json read blocks with load_shard_meta
- Fix mkdtemp leaks in test_lora.py with TemporaryDirectory
2026-07-27 22:34:53 +08:00
ViperEkura
0c86c89af4
refactor : align config field names with Hugging Face
...
- dim -> hidden_size, n_layers -> num_hidden_layers
- dim_ffn -> intermediate_size, n_heads -> num_attention_heads
- n_kv_heads -> num_key_value_heads, max_len -> max_position_embeddings
- norm_eps -> rms_norm_eps, tie_weight -> tie_word_embeddings
- update model, inference, training, scripts, tests, docs
2026-07-20 22:05:31 +08:00
ViperEkura
25d4ea3f91
refactor : 压缩测试代码，消除重复
...
- fixture 替代重复实例化和 tokenizer 落盘
- parametrize 合并同构测试
- helper 消除 save_h5 + DatasetFactory.load 样板
- 净减 272 行
2026-06-19 14:54:39 +08:00
ViperEkura
a4688021bf
feat: 新增LoRA微调模块
...
- LoRALinear基于register_parameter托管base weight，state_dict路径不变
- inject_lora/merge_lora/save_lora/load_lora完备封装
- 24个单元测试覆盖注入、合并、存取、边界场景
2026-05-25 20:15:31 +08:00
ViperEkura
97c7ac0f4f
refactor: Transformer更名为AutoRegressiveLM并新增EmbeddingEncoder
...
- AutoRegressiveLM 注册名改为 autoregressive_lm
- 新增 EmbeddingEncoder 支持 mean/cls/last pooling
- ModelConfig 增加 pooling_type / normalize_embeddings 字段
- 导入、注释、测试全部同步更新
2026-05-17 15:29:20 +08:00
ViperEkura
a44fd22a99
fix: 修复训练与模型参数传递问题
...
- state_dict_fn 传入 CheckpointCallback，修复多卡 DDP 下 key 前缀丢失
- MLA 增加 use_qk_norm 支持，消除参数静默丢失
- moe_topk_method 统一命名为 topk_method
- checkpoint 回调移至最前
2026-05-17 11:20:13 +08:00
ViperEkura
f91bfee33e
refactor: Config序列化统一BaseConfig基类
...
- 新增astrai/config/base.py，提供to_dict/from_dict基类
- 统一命名：load/save → from_file/to_file
- Checkpoint.meta合并训练配置到meta.json
- sys.stderr.warn → warnings.warn
- from_file改为classmethod
2026-05-16 22:06:39 +08:00
ViperEkura
0ba8c70ce1
fix: 修复 MLA 多个 bug 并缩小测试模型参数
...
- MLA kv_b_proj 输出维度和 q_rope 切分偏移修复
- 打通 MLA 配置从 ModelConfig 到 DecoderBlock 的传递路径
- rope_theta 配置不再被忽略，MLA 使用 qk_rope_head_dim
- tie_weight 使用 is True 避免 None 隐式生效
- norm_eps/rope base 类型标注修正
- 测试模型参数缩小 (dim=8, head_dim=4)
- 新增 6 种架构配置 × 2 场景的前向传播测试
2026-05-16 14:57:43 +08:00
ViperEkura
fc278d17ab
feat: 实现模型动态注册机制
2026-04-05 19:38:12 +08:00
ViperEkura
2b26f03bd3
refactor: 拆分engine.py 文件
2026-04-05 00:07:21 +08:00
ViperEkura
861d33b1a1
refactor: 更新inference 部分的实现
2026-04-04 23:49:18 +08:00
ViperEkura
b531232a9b
style: 修改为显式导入
2026-04-04 16:02:49 +08:00
ViperEkura
0852b852f8
refactor: 优化参数传递，清理导入样式
2026-04-03 22:06:32 +08:00
ViperEkura
475de51c7d
feat: 增加server, 并且修改测试单元
2026-04-02 15:05:07 +08:00
ViperEkura
2e009cf59a
chore: 更新项目名称
2026-03-31 09:34:11 +08:00
ViperEkura
426af2d75f
style: 使用ruff 工具优化代码风格
2026-03-30 23:32:28 +08:00
ViperEkura
c01791ff54
feat: 增加推理部分工厂模式
2026-03-30 00:55:15 +08:00
ViperEkura
4da70785b5
refactor(tests): 重构测试文件目录结构
2026-01-08 21:34:52 +08:00
ViperEkura