Repository navigation
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The token-combine and temperature controls missed several layout and boundary cases, and bitmatrix packing dropped assignments after the first 32 top-k slots. This change fixes packing across all tiles and strengthens the three tasks' correctness and timing checks.
apply_temperature(logits, idx_mapping, temperature)signature, rejecting optional extras, defaults, variadic arguments, and changed parameter names or kinds before the harness runs.The original five scored workloads per task, tolerances, warmups, sample counts, and canonical timing helper are unchanged. Added workloads are correctness-only. The packing kernel retains its original single-tile path for the scored shapes; the wrapper's layout normalization is a no-op for those contiguous inputs.
Validation: all three tasks have fresh framework-finalized
PASSreports on AMD Instinct MI355X (gfx950), with all 12 checks passing per task. The same job passed all 52 focused GPU tests with no skips. CPU regression: 2361 passed, 10 GPU-only tests skipped locally.make check-perf-helperspassed. Environment: Linux-6.8.0-107-generic-x86_64-with-glibc2.39, Python 3.12.3, PyTorch 2.11.0+rocm10.0.0, HIP 7.15.26333, Triton 3.8.0; Validator:task_validator. Docker:lmsysorg/sglang@sha256:e20849665c105d389ef91d23c0dc73931aaa6f02056dd10e7b43e4f16c79df69.The paired comparison uses the same packing Triton kernel AST before and after the harness and wrapper changes, with three alternating AB/BA/AB pairs, native graph timing, 10 warmups, and 100 samples. The historical package passes its ten numeric cases and the updated package passes thirteen; the older harness still has the demonstrated validation gap. These measurements check the effect of the harness and layout-normalizing wrapper together; they are not a kernel speedup claim. Qualification is limited to MI355X and the pinned runtime. The checks reject the demonstrated shared-output bypass, but do not prove independent computation for every possible cache-and-copy implementation on identical inputs.
CPU checks:
GPU checks inside the pinned runtime:
The serial Docker validator selects
triton2triton/vllm/triton_combine_sampled_and_draft_tokens,triton2triton/vllm/triton_temperature, andtriton2triton/vllm/triton_pack_bitmatrixthrough the normalmain.pypath. Source and control manifests are checked before and after execution. Each finalized report is matched to the current task sources, initial source inventory, and materialized baseline.