docs: 修正文档中与源码不符的类名、方法签名和模块归属
- Transformer/DecoderBlock/GQA/RotaryEmbedding forward 签名 start_pos → position_ids - _Result → GenerateResult - save_h5/load_h5 从 serialization 移至 dataset 模块 - PagedCache UML 移除内部 PagePool 属性 - 修正 Layer 数不一致(24 vs 32)及 decode 位置分组描述 - 更新文档时间为 2026-05-14
This commit is contained in:
@@ -26,7 +26,9 @@ def processor(
|
||||
model.to(device="cuda", dtype=torch.bfloat16)
|
||||
|
||||
# Create inference engine
|
||||
engine = InferenceEngine(model=model, tokenizer=tokenizer, max_batch_size=batch_size)
|
||||
engine = InferenceEngine(
|
||||
model=model, tokenizer=tokenizer, max_batch_size=batch_size
|
||||
)
|
||||
|
||||
with open(input_json_file, "r", encoding="utf-8") as f:
|
||||
input_data = [json.loads(line) for line in f]
|
||||
|
||||
Reference in New Issue
Block a user