ViperEkura cd14d53707 feat: implement bfd_split packing strategy ...

- BFDSplitPacking splits over-length sequences into chunks before BFD
- All keys (loss_mask, position_ids, ...) split in lockstep for alignment
- No tokens lost vs bfd which truncates over-length sequences
- Tests: token preservation, chunk alignment, short unchanged, vs bfd

2026-07-17 12:38:56 +08:00

..

config

fix : SFT pipeline position_ids default & doc boundary preservation

2026-07-04 15:59:11 +08:00

dataset

feat: support raw JSON files in dataset pipeline and JsonlStore

2026-07-17 12:20:03 +08:00

extension

refactor: stride-based attn interface with layout and causal mask

2026-07-14 21:34:42 +08:00

inference

refactor: clean up inference design patterns

2026-07-05 11:41:54 +08:00

model

perf: replace paged KV cache with contiguous ContiguousCache, decode all groups

2026-07-05 11:34:36 +08:00

parallel

feat : add distributed checkpoint via executor checkpoint_context

2026-07-13 12:27:09 +08:00

preprocessing

feat: implement bfd_split packing strategy

2026-07-17 12:38:56 +08:00

serialization

feat: add --resume flag to decouple weight loading from training resumption

2026-07-16 14:23:23 +08:00

tokenize

refactor : Pipeline 去除去重，ids 重命名为 sequence，泛型透传

2026-05-31 15:14:27 +08:00

trainer

feat: add --resume flag to decouple weight loading from training resumption

2026-07-16 14:23:23 +08:00

__init__.py

chore: bump version to 1.3.9

2026-07-12 21:04:20 +08:00

factory.py

fix: ForwardRef._evaluate Python 3.12 compatibility

2026-07-03 18:41:19 +08:00

protocols.py

refactor: 重构训练后端为 Executor 模式

2026-05-24 20:35:44 +08:00