chore: 简化格式并更新文档
This commit is contained in:
@@ -1,286 +1,147 @@
|
||||
|
||||
|
||||
<div style="display: flex; flex-direction: column; align-items: center; justify-content: center; text-align: center; font-size: 16px; font-weight: bold; margin-top: 50px;">
|
||||
<div align="center">
|
||||
<img src="assets/images/project_logo.png" width="auto" alt="Logo">
|
||||
|
||||
<h1>KHAOSZ</h1>
|
||||
|
||||
<div>
|
||||
<a href="#english" style="text-decoration: none; margin: 0 10px; color: blue;">English</a> |
|
||||
<a href="#chinese" style="text-decoration: none; margin: 0 10px; color: blue;">中文</a>
|
||||
<a href="#english">English</a> •
|
||||
<a href="#chinese">中文</a>
|
||||
</div>
|
||||
|
||||
<h1 style="margin: 20px 0 0 0; font-size: 2.5em; font-weight: bold;">KHAOSZ </h1>
|
||||
|
||||
<p>
|
||||
<strong>A lightweight Transformer training & inference framework</strong>
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<h2 id="english">English Version</h2>
|
||||
## 📖 Table of Contents | 目录
|
||||
|
||||
A training and inference framework for autoregressive Transformer language models.
|
||||
<div align="center">
|
||||
|
||||
**Model Download Options (choose one):**
|
||||
| English | 中文 |
|
||||
|---------|------|
|
||||
| [Installation](#installation) | [安装](#安装) |
|
||||
| [Quick Start](#quick-start) | [快速开始](#快速开始) |
|
||||
| [Documentation](#documentation) | [文档](#文档) |
|
||||
| [License](#license) | [许可证](#许可证) |
|
||||
|
||||
1. Visit [HuggingFace](https://huggingface.co/ViperEk/KHAOSZ) and check **Files and versions**
|
||||
2. Run `scripts/download.py` to download model parameters
|
||||
</div>
|
||||
|
||||
**Demo Video:** [bilibili](https://www.bilibili.com/video/BV1z5RPYHEkd)
|
||||
---
|
||||
|
||||
For training data sources, please refer to the **Model Card** section on the HuggingFace download page.
|
||||
<a id="english"></a>
|
||||
## English
|
||||
|
||||
**License:** The code follows the GPL-3.0 license. Please provide attribution when using it.
|
||||
### Features
|
||||
|
||||
- **📊 Device Selection:** Uses CUDA for training by default
|
||||
- **🌐 Performance Optimization:** Enable `dtype=torch.bfloat16` to accelerate training and reduce memory usage. Ensure your hardware supports this feature
|
||||
- **🤖 Language Support:** The model supports training in Chinese and English. Since the BBPE tokenizer hasn't been trained on multilingual text, OOV (Out-of-Vocabulary) issues are minimal for Chinese and English, but may exist for other languages
|
||||
- 🚀 **High Performance**: Optimized for both training and inference
|
||||
- 🔧 **Flexible**: Support for seq/sft/dpo training
|
||||
- 💡 **Easy to Use**: Simple API with comprehensive examples
|
||||
- 📦 **Lightweight**: Minimal dependencies
|
||||
|
||||
|
||||
### 📌 Training Guide
|
||||
|
||||
To train this Transformer model, follow these steps:
|
||||
|
||||
**(1). Prepare the Dataset:**
|
||||
|
||||
Place the dataset in the specified root directory. This system uses the BBPE tokenizer for tokenization and requires training with pre-tokenized segments (stored as *.h5 format files).
|
||||
|
||||
**(2). Install Dependencies:**
|
||||
### Installation
|
||||
|
||||
```bash
|
||||
git clone https://github.com/username/khaosz.git
|
||||
cd khaosz
|
||||
pip install -e .
|
||||
```
|
||||
|
||||
**(3). Run the Training Script:**
|
||||
### Quick Start
|
||||
|
||||
```bash
|
||||
python train.py \
|
||||
--train_type=train_type[seq, sft, dpo] \
|
||||
--data_root_path=/path/to/dataset \
|
||||
--param_path=/path/to/param_path \
|
||||
--n_epoch=5 \
|
||||
--batch_size=8 \
|
||||
--max_lr=2e-4 \
|
||||
--ckpt_interval=10000 \
|
||||
--ckpt_dir=checkpoints
|
||||
# Train
|
||||
python tools/train.py \
|
||||
--train_type=seq \
|
||||
--data_root_path=/path/to/dataset \
|
||||
--param_path=/path/to/param_path
|
||||
|
||||
# Generate
|
||||
python tools/generate.py --param_path=/path/to/param_path
|
||||
```
|
||||
|
||||
**Parameter Explanation:**
|
||||
- `--train_type`: Training type (seq, sft, dpo)
|
||||
- `--data_root_path`: Dataset root directory
|
||||
- `--param_path`: Path to model training parameters
|
||||
- `--n_epoch`: Total number of training epochs
|
||||
- `--batch_size`: Batch size
|
||||
- `--accumulation_steps`: Number of batches per training step
|
||||
- `--warmup_steps`: Warmup steps
|
||||
- `--max_lr`: Maximum learning rate (using warmup + cosine decay)
|
||||
- `--ckpt_interval`: Checkpoint saving interval
|
||||
- `--ckpt_dir`: Checkpoint saving directory
|
||||
- `--resume_dir`: Resume training from specified path
|
||||
|
||||
|
||||
|
||||
### 👉 Usage Guide
|
||||
|
||||
**(1). Chat with the Model:**
|
||||
|
||||
Open `chat.py` or use the streaming/non-streaming interfaces:
|
||||
|
||||
**Streaming Output:**
|
||||
```python
|
||||
import torch
|
||||
from khaosz import Khaosz
|
||||
|
||||
model_dir = "your_model_parameter_dir"
|
||||
model = Khaosz(model_dir).to(device='cuda', dtype=torch.bfloat16)
|
||||
history = []
|
||||
|
||||
while True:
|
||||
query = input(">> ")
|
||||
if query == "!exit":
|
||||
break
|
||||
|
||||
response_size = 0
|
||||
for response, history in model.stream_generate(
|
||||
query=query,
|
||||
history=history,
|
||||
temperature=0.85,
|
||||
top_p=0.95,
|
||||
top_k=50
|
||||
):
|
||||
print(response[response_size:], end="")
|
||||
response_size = len(response)
|
||||
```
|
||||
|
||||
**Non-streaming Output:**
|
||||
```python
|
||||
import torch
|
||||
from khaosz import Khaosz
|
||||
|
||||
model_dir = "your_model_parameter_dir"
|
||||
model = Khaosz(model_dir).to(device='cuda', dtype=torch.bfloat16)
|
||||
history = []
|
||||
|
||||
while True:
|
||||
query = input(">> ")
|
||||
if query == "!exit":
|
||||
break
|
||||
|
||||
response = model.generate(
|
||||
query=query,
|
||||
history=history,
|
||||
temperature=0.85,
|
||||
top_p=0.95,
|
||||
top_k=50
|
||||
)
|
||||
print(response)
|
||||
```
|
||||
|
||||
**(2). Retrieval-Augmented Generation (RAG):**
|
||||
|
||||
```python
|
||||
import torch
|
||||
from khaosz import Khaosz
|
||||
|
||||
model_dir = "your_model_parameter_dir"
|
||||
model = Khaosz(model_dir).to(device='cuda', dtype=torch.bfloat16)
|
||||
|
||||
retrieved_content = model.retrieve_generate(
|
||||
query=query,
|
||||
retrieve_top_k=5,
|
||||
temperature=0.6,
|
||||
top_k=30,
|
||||
top_p=0.95
|
||||
)
|
||||
print(retrieved_content)
|
||||
```
|
||||
|
||||
<h2 id="chinese">中文版本</h2>
|
||||
这是一个支持基于自回归模式的 Transfomer 语言模型训练以及推理框架
|
||||
|
||||
**模型下载选项（任选其一）：**
|
||||
|
||||
1. 访问 [HuggingFace](https://huggingface.co/ViperEk/KHAOSZ) 查看 **Files and versions**
|
||||
2. 运行 `scripts/download.py` 下载模型参数
|
||||
|
||||
**演示视频：** [bilibili](https://www.bilibili.com/video/BV1z5RPYHEkd)
|
||||
|
||||
训练数据来源请参见 HuggingFace 下载页面中的 **Model Card** 部分。
|
||||
|
||||
**许可证：** 代码遵循 GPL-3.0 协议，使用时请注明出处。
|
||||
|
||||
- **📊 设备选择：** 默认使用 CUDA 进行训练
|
||||
- **🌐 性能优化：** 启用 `dtype=torch.bfloat16` 以加速训练并减少内存占用，请确保硬件支持该特性
|
||||
- **🤖 语言支持：** 模型支持中文和英文训练。由于 BBPE 分词器未使用多语言文本训练，因此中英文的 OOV（未登录词）问题较少，其他语言可能存在 OOV 问题
|
||||
|
||||
|
||||
### 📌 训练指南
|
||||
|
||||
要训练该 Transformer 模型，请按照以下步骤操作：
|
||||
|
||||
**(1). 准备数据集：**
|
||||
|
||||
将数据集放置在指定的根目录下， 本系统采用 BBPE 分词器进行分词，并且要求使用已经经过分词的 token 分段训练（分段存储为 *.h5 格式）
|
||||
|
||||
**(2). 安装依赖：**
|
||||
### Demo
|
||||
|
||||
```bash
|
||||
# run download before using
|
||||
python demo/download.py
|
||||
|
||||
# run demo
|
||||
python demo/stream_chat.py
|
||||
python demo/generate_batch.py
|
||||
python demo/generate_ar.py
|
||||
```
|
||||
|
||||
- [bilibili](https://www.bilibili.com/video/BV1z5RPYHEkd)
|
||||
|
||||
<a id="license"></a>
|
||||
### License
|
||||
|
||||
GPL-3.0
|
||||
|
||||
---
|
||||
|
||||
<a id="chinese"></a>
|
||||
## 中文
|
||||
|
||||
### 特性
|
||||
|
||||
- 🚀 **高性能**: 训练与推理双向优化
|
||||
- 🔧 **灵活**: 支持 seq/sft/dpo 多种训练方式
|
||||
- 💡 **易用**: 简洁的 API 与丰富的示例
|
||||
- 📦 **轻量**: 依赖少，部署简单
|
||||
|
||||
### 安装
|
||||
|
||||
```bash
|
||||
git clone https://github.com/username/khaosz.git
|
||||
cd khaosz
|
||||
pip install -e .
|
||||
```
|
||||
|
||||
**(3). 运行训练脚本：**
|
||||
### 快速开始
|
||||
|
||||
```bash
|
||||
python train.py \
|
||||
--train_type=train_type[seq, sft, dpo] \
|
||||
--data_root_path=/path/to/dataset \
|
||||
--param_path=/path/to/param_path \
|
||||
--n_epoch=5 \
|
||||
--batch_size=8 \
|
||||
--max_lr=2e-4 \
|
||||
--ckpt_interval=10000 \
|
||||
--ckpt_dir=checkpoints
|
||||
# 训练
|
||||
python tools/train.py \
|
||||
--train_type=seq \
|
||||
--data_root_path=/path/to/dataset \
|
||||
--param_path=/path/to/param_path
|
||||
|
||||
# 生成
|
||||
python tools/generate.py --param_path=/path/to/param_path
|
||||
```
|
||||
|
||||
**参数说明：**
|
||||
- `--train_type`: 训练类型（seq, sft, dpo）
|
||||
- `--data_root_path`: 数据集根目录
|
||||
- `--param_path`: 模型训练参数路径
|
||||
- `--n_epoch`: 总训练轮数
|
||||
- `--batch_size`: 批量大小
|
||||
- `--accumulation_steps`: 每个训练步骤的 batch 数量
|
||||
- `--warmup_steps`: 预热步数（warmup steps）
|
||||
- `--max_lr`: 最大学习率（使用预热 + 余弦衰减）
|
||||
- `--ckpt_interval`: 检查点保存间隔
|
||||
- `--ckpt_dir`: 检查点保存目录
|
||||
- `--resume_dir`: 从指定路径恢复训练
|
||||
### 演示
|
||||
|
||||
```bash
|
||||
# 使用前先下载模型
|
||||
python demo/download.py
|
||||
|
||||
|
||||
### 👉 使用指南
|
||||
|
||||
**(1). 与模型对话：**
|
||||
|
||||
打开 `chat.py` 或使用流式/非流式接口：
|
||||
|
||||
**流式输出：**
|
||||
```python
|
||||
import torch
|
||||
from khaosz import Khaosz
|
||||
|
||||
model_dir = "your_model_parameter_dir"
|
||||
model = Khaosz(model_dir).to(device='cuda', dtype=torch.bfloat16)
|
||||
history = []
|
||||
|
||||
while True:
|
||||
query = input(">> ")
|
||||
if query == "!exit":
|
||||
break
|
||||
|
||||
response_size = 0
|
||||
for response, history in model.stream_generate(
|
||||
query=query,
|
||||
history=history,
|
||||
temperature=0.85,
|
||||
top_p=0.95,
|
||||
top_k=50
|
||||
):
|
||||
print(response[response_size:], end="")
|
||||
response_size = len(response)
|
||||
# 运行示例
|
||||
python demo/stream_chat.py
|
||||
python demo/generate_batch.py
|
||||
python demo/generate_ar.py
|
||||
```
|
||||
|
||||
**非流式输出：**
|
||||
```python
|
||||
import torch
|
||||
from khaosz import Khaosz
|
||||
- [bilibili](https://www.bilibili.com/video/BV1z5RPYHEkd)
|
||||
|
||||
model_dir = "your_model_parameter_dir"
|
||||
model = Khaosz(model_dir).to(device='cuda', dtype=torch.bfloat16)
|
||||
history = []
|
||||
### 许可证
|
||||
|
||||
while True:
|
||||
query = input(">> ")
|
||||
if query == "!exit":
|
||||
break
|
||||
|
||||
response = model.generate(
|
||||
query=query,
|
||||
history=history,
|
||||
temperature=0.85,
|
||||
top_p=0.95,
|
||||
top_k=50
|
||||
)
|
||||
print(response)
|
||||
```
|
||||
GPL-3.0
|
||||
|
||||
**(2). 基于检索的生成（RAG）：**
|
||||
---
|
||||
|
||||
```python
|
||||
import torch
|
||||
from khaosz import Khaosz
|
||||
<a id="documentation"></a>
|
||||
## 📚 Documentation | 文档
|
||||
|
||||
model_dir = "your_model_parameter_dir"
|
||||
model = Khaosz(model_dir).to(device='cuda', dtype=torch.bfloat16)
|
||||
| Document | 说明 |
|
||||
|----------|------|
|
||||
| [参数说明](assets/docs/params.md) | Training & inference parameters |
|
||||
| [设计文档](assets/docs/design.md) | Framework design |
|
||||
| [数据流程](assets/docs/dataflow.md) | Data processing pipeline |
|
||||
| [模型介绍](assets/docs/introduction.md) | Model architecture |
|
||||
|
||||
retrieved_content = model.retrieve_generate(
|
||||
query=query,
|
||||
retrieve_top_k=5,
|
||||
temperature=0.6,
|
||||
top_k=30,
|
||||
top_p=0.95
|
||||
)
|
||||
print(retrieved_content)
|
||||
```
|
||||
### Download | 下载
|
||||
|
||||
- [HuggingFace](https://huggingface.co/ViperEk/KHAOSZ)
|
||||
- `python demo/download.py`
|
||||
Reference in New Issue
Block a user