- replace the O(batch*vocab) count materialization and boolean-mask/unique indexing in FrequencyPenaltyStrategy with a flat-bucket where + index_add_ + elementwise subtraction that never forces a device-host synchronization
- the hidden nonzero syncs inside masked indexing and torch.unique dominated the old path under GPU contention, not the scatter itself
- semantics unchanged (per-row penalties, padding mask, zero-penalty skip); all 27 sampling tests pass
- Benchmark: L20 SM89, batch 8 vocab 100k, penalty overhead 6.3ms to 47us and full sampling pipeline 7196us to 969us.
ViperEkura
ce2f9d13b3