- Storage Backends: add JsonlStore (lazy on-the-fly tokenization) - Describe memory-efficiency tradeoff across three backends - Fix length filter: per-text-field skip, not per-sample global - Abstract: HDF5/mmap -> tiered storage backends - Rename metric figures (singular) and update references
92 KiB
1500x380px
92 KiB
1500x380px