ViperEkura ccf728a1b7 perf: eliminate GPU syncs in contiguous cache write/gather hot paths ...

- Replace .tolist() calls with _total_len in gather(); move _slot_len updates from per-layer write to once-per-step bind_tasks

- Use torch.as_tensor instead of torch.tensor in decode penalty history construction

2026-07-21 16:40:56 +08:00

..

api

fix: memmap mode=r, tool parser json.loads, greedy decode

2026-07-19 16:38:28 +08:00

core

perf: eliminate GPU syncs in contiguous cache write/gather hot paths

2026-07-21 16:40:56 +08:00

__init__.py

feat: add frequency penalty to inference sampling pipeline

2026-07-17 21:28:31 +08:00

engine.py

fix: memmap mode=r, tool parser json.loads, greedy decode

2026-07-19 16:38:28 +08:00

sample.py

refactor: simplify sample return_logprobs path

2026-07-20 13:16:18 +08:00