ViperEkura a95794d3db perf: use Rust-native DecodeStream for O(n) streaming decode ...

- Replace hand-rolled StreamDecoder (O(n^2) full-history re-decode per token) with tokenizers.decoders.DecodeStream
- Keep O(1) bounded token buffer internally via prefix drain instead of accumulating all token IDs
- Simplify flush_remaining to no-op since stream always emits completed text per step
- Benchmark on 8000 tokens: 2305ms -> 3.9ms (~592x speedup)

2026-07-28 14:32:10 +08:00

..

api

refactor : merge max_prompt_len into max_seq_len, replace assert with raise

2026-07-27 08:05:11 +08:00

core

perf: use Rust-native DecodeStream for O(n) streaming decode

2026-07-28 14:32:10 +08:00

__init__.py

feat: add frequency penalty to inference sampling pipeline

2026-07-17 21:28:31 +08:00

engine.py

refactor : merge max_prompt_len into max_seq_len, replace assert with raise

2026-07-27 08:05:11 +08:00

sample.py

refactor: simplify sample return_logprobs path

2026-07-20 13:16:18 +08:00