refactor : BaseConfig 提供 from_json/to_json,嵌套 config 自动反序列化

- from_json/to_json 上提至 BaseConfig,所有子类自动继承
- _coerce 新增 dict 到 BaseConfig 子类的递归反序列化,消除子类 from_dict 重载
- PipelineConfig 等子类仅声明字段,零样板代码
- 测试 tokenizer 改为自包含 BPE(含 chat template),不依赖 params/ 目录
- 特殊 token 改用 ASCII 字符,兼容所有平台
This commit is contained in:
2026-05-30 21:04:19 +08:00
parent 69207e2c57
commit 31ae2deeba
3 changed files with 123 additions and 75 deletions
+13 -1
View File
@@ -1,6 +1,7 @@
import json
from dataclasses import MISSING, dataclass, fields
from typing import Any, Dict, Optional, Self, get_type_hints
from pathlib import Path
from typing import Any, Dict, Optional, Self, Union, get_type_hints
@dataclass
@@ -83,4 +84,15 @@ class BaseConfig:
return value
if isinstance(value, target_type):
return value
if isinstance(value, dict) and issubclass(target_type, BaseConfig):
return target_type.from_dict(value)
raise TypeError
@classmethod
def from_json(cls, path: Union[str, Path]) -> Self:
with open(path, "r", encoding="utf-8") as f:
return cls.from_dict(json.load(f))
def to_json(self, path: Union[str, Path]):
with open(path, "w", encoding="utf-8") as f:
json.dump(self.to_dict(), f, indent=2, ensure_ascii=False)