docs: 修正文档中与源码不符的类名、方法签名和模块归属
- Transformer/DecoderBlock/GQA/RotaryEmbedding forward 签名 start_pos → position_ids - _Result → GenerateResult - save_h5/load_h5 从 serialization 移至 dataset 模块 - PagedCache UML 移除内部 PagePool 属性 - 修正 Layer 数不一致(24 vs 32)及 decode 位置分组描述 - 更新文档时间为 2026-05-14
This commit is contained in:
@@ -12,7 +12,7 @@ AstrAI adopts a modular design with the following main components:
|
||||
- **Config Module** (`astrai/config/`): ModelConfig, TrainConfig
|
||||
- **Factory Module** (`astrai/factory/`): Registry, BaseFactory for component registration
|
||||
- **Parallel Module** (`astrai/parallel/`): Distributed training support
|
||||
- **Serialization** (`astrai/serialization.py`): HDF5 data loading, checkpoint management
|
||||
- **Serialization** (`astrai/serialization.py`): Checkpoint management with safetensors
|
||||
|
||||
## Data Flow Diagram
|
||||
|
||||
@@ -59,7 +59,7 @@ flowchart LR
|
||||
|
||||
## Detailed Module Descriptions
|
||||
|
||||
### 1. Serialization (`astrai/serialization.py`)
|
||||
### 1. Data Serialization (`astrai/dataset/storage.py` & `astrai/serialization.py`)
|
||||
|
||||
- **`save_h5`**: Saves tensors by groups as HDF5 files (`.h5`), each key maps to a list of tensors
|
||||
- **`load_h5`**: Loads `.h5` files, returns `Dict[str, List[Tensor]]`, supports shared memory
|
||||
@@ -234,4 +234,4 @@ Background thread runs continuously:
|
||||
- **Inference Loading**: `AutoModel.from_pretrained()` loads from the same safetensors format.
|
||||
- **Dataset Serialization**: HDF5 with shared memory support for large-scale pre-training data.
|
||||
|
||||
> Document Update Time: 2026-05-09
|
||||
> Document Update Time: 2026-05-14
|
||||
|
||||
Reference in New Issue
Block a user