0z5a 01bcd0d105 perf: batch ragged prefill requests ...

- Pack prompts with a shared prefix start and attention backend into one forward.

- Select per-request final logits from cumulative query lengths.

- Cover ragged tokens, logprobs, scheduling, and documentation.

2026-09-02 15:00:16 +08:00

..

data

refactor: unify CUDA skip guards in tests

2026-08-22 21:08:06 +08:00

extension

refactor: accept arbitrary K in bf16 gemv with aligned head-tail sweeps

2026-09-02 14:48:01 +08:00

inference

perf: batch ragged prefill requests

2026-09-02 15:00:16 +08:00

module

test: refactor tests and fix inference edge cases

2026-08-21 22:59:51 +08:00

optim

refactor: unify CUDA skip guards in tests

2026-08-22 21:08:06 +08:00

parallel

style: fix ruff lint warnings

2026-08-07 14:17:48 +08:00

serialization

test: refactor tests and fix inference edge cases

2026-08-21 22:59:51 +08:00

trainer

fix: report failed rollout requests

2026-09-02 12:52:50 +08:00

conftest.py

perf: pure FP8 fwd/bwd and lean non-transposed GEMM

2026-08-23 15:38:30 +08:00

helpers.py

refactor: deduplicate and restructure test suite

2026-08-20 01:53:28 +08:00

test_serve_cli.py

feat: unify Docker serving configuration in YAML

2026-08-21 23:16:45 +08:00

test_serve_runtime.py

refactor: move runtime parsers into scripts/docker

2026-08-27 12:33:32 +08:00

test_train_cli.py

Make Nora+NAdamW the default optimizer

2026-07-31 23:16:39 +08:00