ViperEkura/AstrAI
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Commit Graph

  • d28552f878 refactor: use C++17 struct dispatch in csrc tests, tighten paged tolerances to 0.01 ViperEkura 2026-08-09 10:23:12 +08:00
  • be90dfe2bd fix: isolate continuous batch decode state ViperEkura 2026-08-09 00:56:54 +08:00
  • a33ca04f60 fix: synchronize final decode async copy ViperEkura 2026-08-09 00:31:47 +08:00
  • 7f0e8bb8c2 fix: let flash backend handle 4D causal prefill mask ViperEkura 2026-08-08 23:50:59 +08:00
  • 0c1b7664c1 refactor: split infer core into subpackages by concern ViperEkura 2026-08-08 23:22:37 +08:00
  • 3fa7e66676 refactor: decouple task cache from PagePool and unify steady-state detection ViperEkura 2026-08-08 22:43:06 +08:00
  • ca50fe4721 refactor: remove inference redundancy and fix cache leaks ViperEkura 2026-08-08 21:45:01 +08:00
  • d9240ab149 refactor: split train context build steps ViperEkura 2026-08-08 17:14:26 +08:00
  • d7cd69fef5 feat: add streaming IterableDataset for pretraining ViperEkura 2026-08-08 16:15:10 +08:00
  • 9bff61fb91 perf: use cudaEvent for precise GPU timing in debug logs ViperEkura 2026-08-08 13:18:11 +08:00
  • 0b661bae85 fix: remove blocking cleanup from streaming generator ViperEkura 2026-08-08 13:12:40 +08:00
  • ae9fd546ef perf: merge prefill warmup into _warmup_cuda_graphs ViperEkura 2026-08-08 13:05:46 +08:00
  • e3ea850dc9 fix: default backend race, raise on explicit fallback ViperEkura 2026-08-08 12:59:47 +08:00
  • 6e5088cc7d refactor: remove prefill from CUDA graph warmup ViperEkura 2026-08-08 12:48:45 +08:00
  • cbc584470d refactor: centralize logging in astrai.logging, replace ASTRAI_TIMED with log level ViperEkura 2026-08-08 12:32:51 +08:00
  • cb60713a72 feat: add per-task throughput and latency metrics ViperEkura 2026-08-08 12:07:08 +08:00
  • c52a2487ae fix: rename CUDA wheels with tag suffix to avoid upload clash ViperEkura 2026-08-08 00:48:43 +08:00
  • 49aaa9a714 version: bump to 1.3.13 ViperEkura 2026-08-07 23:53:54 +08:00
  • 056c1382ff docs: sync all documentation with current codebase ViperEkura 2026-08-07 23:52:27 +08:00
  • f163520fff refactor: break JsonlStore→preprocessing circular dependency ViperEkura 2026-08-07 23:22:32 +08:00
  • 1b1f1a0707 fix: add dtype guard to FlashAttnBackend capability check ViperEkura 2026-08-07 23:07:43 +08:00
  • 184fbbce5c refactor: extract shared steady-state increment detection ViperEkura 2026-08-07 23:00:25 +08:00
  • 02469887f5 refactor: simplify inference engine and backend dispatch ViperEkura 2026-08-07 22:18:47 +08:00
  • 05739629fc feat: add timed() context manager and backend supports() ViperEkura 2026-08-07 20:48:14 +08:00
  • e0f7fa8e13 feat: enable CUDA graph by default with init-time warmup ViperEkura 2026-08-07 19:56:21 +08:00
  • af25833fab fix: add out_buf to attn_paged_decode for CUDA graph capture compatibility ViperEkura 2026-08-07 19:45:59 +08:00
  • 6572be4f98 fix: prevent signal handler test from racing with training completion ViperEkura 2026-08-07 18:30:35 +08:00
  • 81788faef4 perf: use flash_attn_with_kvcache for contiguous cache decode ViperEkura 2026-08-07 18:19:17 +08:00
  • 0e7fe57d96 fix: use max_context_len for stable num_splits in paged decode - PagedKV::host_kv_len now returns max_context_len instead of max_seq_len - Eliminates grid-z instability for CUDA graph capture/replay - Restore skip_no_kernel re-export accidentally removed by ruff --fix ViperEkura 2026-08-07 14:42:53 +08:00
  • 55ee258e95 style: fix ruff lint warnings ViperEkura 2026-08-07 14:16:55 +08:00
  • ef1bb6f401 refactor: unify greedy check with _is_greedy helper ViperEkura 2026-08-06 21:14:04 +08:00
  • 6f49738991 feat: auto-select best available attention backend ViperEkura 2026-08-07 13:48:59 +08:00
  • a59ae8f32e fix: use c10::optional for o_part_buf/ml_part_buf decode kernel params ViperEkura 2026-08-06 20:50:48 +08:00
  • 6054b8dbd4 feat: add CUDA-graph capture for decode forward ViperEkura 2026-08-06 19:56:59 +08:00
  • 6f67ba8942 perf: move decode split partials to InferenceWorkspace ViperEkura 2026-08-06 19:07:26 +08:00
  • d0c5debbab perf: preload V in decode split-kv shared mem and cache partial tensors ViperEkura 2026-08-06 18:26:46 +08:00
  • 4f2e03880b fix : repair and extend throughput benchmark ViperEkura 2026-08-06 12:51:30 +08:00
  • 5c180cfa90 fix : handle zero-token batch generation ViperEkura 2026-08-06 12:31:09 +08:00
  • 6f09b1d2ee docs : clarify radix cache architecture ViperEkura 2026-08-06 11:50:45 +08:00
  • b2230fefd8 feat : add radix prefix cache ViperEkura 2026-08-06 11:44:49 +08:00
  • 654e6eb0d1 fix : correct prefill sampling and record alignment ViperEkura 2026-08-05 22:14:48 +08:00
  • a317a4756b refactor: stateless MoE routing with grouped dispatch ViperEkura 2026-08-05 18:39:41 +08:00
  • 9b7e6c205f feat: add moe auxloss and metrics ViperEkura 2026-08-05 18:12:28 +08:00
  • 602b5ce216 docs : add project capability overview ViperEkura 2026-08-05 15:47:42 +08:00
  • 8152760b5f refactor : use factory for attention backends ViperEkura 2026-08-05 15:37:22 +08:00
  • 8c052c99ee feat: add optional FlashAttention (FA2/FA3) backend ViperEkura 2026-08-05 15:27:26 +08:00
  • 2667b8116d refactor: unify paged and contiguous attention kernels via KVSource policy ViperEkura 2026-08-05 14:06:04 +08:00
  • 6dffb0305a fix: satisfy ruff format and import lint in setup.py ViperEkura 2026-08-04 21:32:33 +08:00
  • 49a9c6b3d2 build: migrate CUDA kernel build to CMake ViperEkura 2026-08-04 21:21:28 +08:00
  • cdf9145ecf docs: align CUDA kernel and RoPE docs with code ViperEkura 2026-08-03 20:54:40 +08:00
  • 85f0461b3b docs: update license refs from GPL-3.0 to Apache-2.0 ViperEkura 2026-08-03 20:21:36 +08:00
  • 9f0e9195f7 Update LICENSE ViperEkura 2026-08-03 20:18:27 +08:00
  • 88751d0b08 refactor: share prefill+decode step between scheduler paths ViperEkura 2026-08-03 13:45:27 +08:00
  • d0e5d910de perf: reduce remaining per-step allocations ViperEkura 2026-08-03 01:10:06 +08:00
  • a03504a280 perf: preallocate inference decode buffers ViperEkura 2026-08-03 00:55:26 +08:00
  • d033b2ef0f perf: cache per-step decode tensor construction ViperEkura 2026-08-02 20:32:13 +08:00
  • 8447f88f61 fix: size KV pool from prompt/gen args in benchmark ViperEkura 2026-08-02 16:22:41 +08:00
  • b1b65a657e perf: target 512 grid blocks for decode split-K ViperEkura 2026-08-02 16:05:56 +08:00
  • 3439e3104e perf: launch CUDA kernels on torch's current stream ViperEkura 2026-08-02 13:18:41 +08:00
  • 288ba20db1 docs: audit non-CUDA documentation ViperEkura 2026-08-02 07:39:24 +08:00
  • 020e2eff4e refactor: emit strategy metrics as floats ViperEkura 2026-08-02 06:38:28 +08:00
  • 1c7369f293 feat: add MoE auxiliary loss metrics ViperEkura 2026-08-02 06:12:04 +08:00
  • 0fc1b1bd46 feat: extend DeepSeek MoE configuration ViperEkura 2026-08-02 05:30:40 +08:00
  • d7db37a70f fix: preserve MoE routing defaults ViperEkura 2026-08-02 05:30:26 +08:00
  • 6d98bb4f9f 20260801-moe model impl Gaolingx 2026-08-01 22:48:58 +08:00
  • 925cbedc93 feat: scalar paged prefill fallback and decode causal fix ViperEkura 2026-08-01 16:50:35 +08:00
  • fda82ee232 perf: drop redundant smem zero-init in paged decode kernel ViperEkura 2026-08-01 16:17:34 +08:00
  • 4b25664c79 perf: precompute kv_indptr once per decode step ViperEkura 2026-08-01 16:09:26 +08:00
  • a27c8a819d test: prune low-value and duplicate tests ViperEkura 2026-08-01 16:01:20 +08:00
  • 91acaf4b0b refactor: unify attention mask to single attn_mask tensor ViperEkura 2026-08-01 15:49:26 +08:00
  • 41dcf0feb9 feat: SGLang-style paged attention kernels replace page-table path ViperEkura 2026-08-01 15:40:44 +08:00
  • 9960f79920 feat: parallel kernel build via BUILD_PARALLEL env var ViperEkura 2026-08-01 11:26:39 +08:00
  • 7feeb0b93e refactor: replace magic layout ints with TensorLayout enum ViperEkura 2026-08-01 11:04:45 +08:00
  • 3639b50b4a chore: bump version to 1.3.12 v1.3.12 ViperEkura 2026-08-01 09:22:16 +08:00
  • d855c09cf3 fix: use torch.optim.AdamW in ManoAdamW instead of NAdamW ViperEkura 2026-08-01 09:20:54 +08:00
  • d6bfb09863 feat: add grad_snr metric with EMA-based gradient SNR tracking ViperEkura 2026-08-01 08:54:44 +08:00
  • 6db276f37a feat: add Mano manifold optimizer (mano_adamw) ViperEkura 2026-08-01 08:51:08 +08:00
  • 6c76c16480 feat: group train CLI options in --help output ViperEkura 2026-08-01 08:32:28 +08:00
  • 11073bd1d2 refactor: extract composite optimizer helpers and unify naming ViperEkura 2026-08-01 08:07:45 +08:00
  • 25c9e81b2b refactor: keep muon_adamw as default optimizer and drop nora docs ViperEkura 2026-08-01 07:50:53 +08:00
  • ffbd9b57c9 Merge branch 'codex/nora-nadamw-default' into experiment ViperEkura 2026-08-01 07:49:30 +08:00
  • 04899a2b15 Make Nora+NAdamW the default optimizer QueenAmish 2026-07-31 23:16:39 +08:00
  • 530d280e33 perf: remove split partials memset and overlap decode tile loads - alloc_split_partials now uses torch::empty: the split kernel writes every slot it owns, so the per-call zeros/full memset was pure overhead (2 kernels per layer per step) - decode split-KV MMA kernels now run a true multi-stage cp.async pipeline (wait_group<STAGES-1> instead of wait_group<0>), keeping STAGES-1 tile loads in flight; the old wait_group<0> serialized load and compute so deeper STAGES made no difference - add a fallback path when ntiles < STAGES to avoid a race on the last tile ViperEkura 2026-07-31 22:37:44 +08:00
  • 21ddead238 fix: stabilize paged decode attention kernels ViperEkura 2026-07-31 21:00:01 +08:00
  • 7aa5ed09d9 refactor: unify rotary embedding interface and update docs ViperEkura 2026-07-31 16:52:25 +08:00
  • 75411ce0cc fix: skip CUDA rotary kernel when grad is enabled ViperEkura 2026-07-31 15:40:38 +08:00
  • 9f83d982ec refactor: move compiled kernel .so files into extension/lib ViperEkura 2026-07-31 15:34:32 +08:00
  • 3e67b4f88d perf: add fused CUDA rotary embedding kernel ViperEkura 2026-07-31 15:27:31 +08:00
  • 50cfd0d555 perf: reduce decode overhead in scheduler and executor ViperEkura 2026-07-31 14:46:03 +08:00
  • 5756054d38 build: parametrize CUDA version for wheels and docker ViperEkura 2026-07-31 14:02:57 +08:00
  • 738cb8f128 fix: broadcast ref/old model state_dict for FSDP ViperEkura 2026-07-31 08:32:22 +08:00
  • 28d1bd07cf style: unify decode expf to __expf ViperEkura 2026-07-31 00:19:18 +08:00
  • 02625739fe perf: increase eval batch sizes and add max_seq_len ViperEkura 2026-07-30 23:55:37 +08:00
  • f688cd9c5a fix: update benchmark to use checkpoint loading and CudaBackend ViperEkura 2026-07-30 22:54:45 +08:00
  • 8055027df7 perf: enable paged MMA kernel for page_size=1 ViperEkura 2026-07-30 22:06:41 +08:00
  • 3067a8e1a6 feat: unify attention backend with multi-dim mask support ViperEkura 2026-07-30 20:38:34 +08:00
  • 97114b95a4 docs: update for attention backend and extension API ViperEkura 2026-07-30 18:50:16 +08:00
  • 32fd03a025 feat: add CudaBackend and rename to fwd_decode/fwd_prefill ViperEkura 2026-07-30 18:45:33 +08:00
  • 21bf37dd83 refactor: unify extension API to blhd layout and is_causal ViperEkura 2026-07-30 18:34:51 +08:00
  • 5b67d5865a feat: add AttentionBackend ABC with context manager ViperEkura 2026-07-30 18:11:16 +08:00