refactor: replace diamond inheritance with mixin composition
- StreamStore/RecordStore → Streamable/Recordable (stateless mixins) - Store is sole base class, no MRO ambiguity - H5Store/MmapStore/JsonlStore mix in both traits explicitly - segments_are_records declared per-subclass (H5/Jsonl=True, bin=False) - Add tests for dpo_tokenize, lazy jsonl, dual-mode H5, stream-only bin - Remove unused _to_tensor helper
This commit is contained in:
@@ -9,10 +9,10 @@ from astrai.dataset.storage import (
|
||||
H5Store,
|
||||
JsonlStore,
|
||||
MmapStore,
|
||||
RecordStore,
|
||||
Recordable,
|
||||
Store,
|
||||
StoreFactory,
|
||||
StreamStore,
|
||||
Streamable,
|
||||
detect_format,
|
||||
)
|
||||
from astrai.serialization import (
|
||||
@@ -28,8 +28,8 @@ __all__ = [
|
||||
"dpo_collate_fn",
|
||||
"grpo_collate_fn",
|
||||
"Store",
|
||||
"StreamStore",
|
||||
"RecordStore",
|
||||
"Streamable",
|
||||
"Recordable",
|
||||
"StoreFactory",
|
||||
"H5Store",
|
||||
"MmapStore",
|
||||
|
||||
Reference in New Issue
Block a user