ViperEkura 3406157431 refactor: standardize packed 3d inference ...

- keep training attention on dense 4d tensors
- use packed 3d tensors with KV cache for inference
- extend CUDA rotary embedding to packed 3d inputs
- adapt torch, CUDA and FlashAttention backend dispatch

2026-08-16 13:24:02 +08:00

..

cache

refactor: standardize packed 3d inference

2026-08-16 13:24:02 +08:00

network

refactor: split infer core into subpackages by concern

2026-08-08 23:43:05 +08:00

runtime

refactor: standardize packed 3d inference

2026-08-16 13:24:02 +08:00

__init__.py

refactor: split infer core into subpackages by concern

2026-08-08 23:43:05 +08:00

engine.py

feat: propagate attention backend across scheduler threads

2026-08-09 13:32:40 +08:00

metrics.py

refactor: split infer core into subpackages by concern

2026-08-08 23:43:05 +08:00

scheduler.py

feat: propagate attention backend across scheduler threads

2026-08-09 13:32:40 +08:00

task.py

feat: propagate attention backend across scheduler threads

2026-08-09 13:32:40 +08:00

workspace.py

perf: use int32 paged KV indices

2026-08-15 13:17:06 +08:00