- detect_format now recognizes .json directories as jsonl store - JsonlStore loads .json arrays and dicts alongside .jsonl - tokenizer_path defaults to dataset dir when omitted - Pipeline._iter_items handles .json files (arrays/single dict) - Tests: detect_format, seq load, self-contained dataset dir |
||
|---|---|---|
| .. | ||
| data | ||
| inference | ||
| module | ||
| parallel | ||
| trainer | ||
| conftest.py | ||