Skip to content

[Klaud Cold] Update qwen3.5-fp4-gb200-dynamo-sglang-mtp SGLang image to v0.5.21-cu130 / [Klaud Cold] 将 qwen3.5-fp4-gb200-dynamo-sglang-mtp 的 SGLang 镜像更新至 v0.5.21-cu130 #4090

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Klaud-Cold wants to merge 3 commits into from
Closed

[Klaud Cold] Update qwen3.5-fp4-gb200-dynamo-sglang-mtp SGLang image to v0.5.21-cu130 / [Klaud Cold] 将 qwen3.5-fp4-gb200-dynamo-sglang-mtp 的 SGLang 镜像更新至 v0.5.21-cu130 #4090

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
078f128
chore(qwen3.5-fp4-gb200-dynamo-sglang-mtp): update SGLang image to v0…
github-actions[bot] Oct 15, 2026
f182efb
fix(qwen3.5-fp4-gb200-dynamo-sglang-mtp): use the image's Dynamo for …
github-actions[bot] Oct 15, 2026
57441f3
fix(qwen3.5-fp4-gb200-dynamo-sglang-mtp): keep prefill CUDA graphs di…
github-actions[bot] Oct 15, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 7 additions & 6 deletions ...ks/multi_node/srt-slurm-recipes/qwen3.5/sglang/gb200-fp4-v2/8k1k/mtp/1p1d-tp4-tp4-c1.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ roles:
SGLANG_DISAGGREGATION_WAITING_TIMEOUT: '100000'
SGLANG_MOONCAKE_CUSTOM_MEM_POOL: 'True'
SGLANG_USE_MESSAGE_QUEUE_BROADCASTER: '0'
SGLANG_DISABLE_TP_MEMORY_INBALANCE_CHECK: '1'
SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK: '0'
SGLANG_HEALTH_CHECK_TIMEOUT: '3600'
SGLANG_HEALTH_STARTING_OK: '1'
SGLANG_ENABLE_HEALTH_ENDPOINT_GENERATION: '0'
Expand All @@ -54,7 +54,7 @@ roles:
tensor-parallel-size: 4
data-parallel-size: 1
expert-parallel-size: 1
mamba-scheduler-strategy: no_buffer
mamba-radix-cache-strategy: no_buffer
mamba-ssm-dtype: bfloat16
mamba-track-interval: 2048
attention-backend: trtllm_mha
Expand All @@ -69,7 +69,8 @@ roles:
context-length: 9236
max-total-tokens: 128000
max-running-requests: 128
cuda-graph-max-bs: 4
cuda-graph-max-bs-decode: 4
cuda-graph-backend-prefill: disabled
chunked-prefill-size: 32768
max-prefill-tokens: 32768
scheduler-recv-interval: 10
Expand Down Expand Up @@ -106,7 +107,7 @@ roles:
SGLANG_DECODE_BOOTSTRAP_TIMEOUT: '1000'
SGLANG_MOONCAKE_CUSTOM_MEM_POOL: 'True'
SGLANG_USE_MESSAGE_QUEUE_BROADCASTER: '0'
SGLANG_DISABLE_TP_MEMORY_INBALANCE_CHECK: '1'
SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK: '0'
SGLANG_HEALTH_CHECK_TIMEOUT: '3600'
SGLANG_HEALTH_STARTING_OK: '1'
SGLANG_ENABLE_HEALTH_ENDPOINT_GENERATION: '0'
Expand All @@ -122,7 +123,7 @@ roles:
tensor-parallel-size: 4
data-parallel-size: 1
expert-parallel-size: 1
mamba-scheduler-strategy: no_buffer
mamba-radix-cache-strategy: no_buffer
mamba-ssm-dtype: bfloat16
mamba-track-interval: 128
attention-backend: trtllm_mha
Expand All @@ -142,7 +143,7 @@ roles:
max-total-tokens: 1500000
max-mamba-cache-size: 256
max-running-requests: 128
cuda-graph-max-bs: 256
cuda-graph-max-bs-decode: 256
chunked-prefill-size: 32768
max-prefill-tokens: 32768
scheduler-recv-interval: 10
Expand Down
13 changes: 7 additions & 6 deletions ...ks/multi_node/srt-slurm-recipes/qwen3.5/sglang/gb200-fp4-v2/8k1k/mtp/1p1d-tp4-tp4-c4.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ roles:
SGLANG_DISAGGREGATION_WAITING_TIMEOUT: '100000'
SGLANG_MOONCAKE_CUSTOM_MEM_POOL: 'True'
SGLANG_USE_MESSAGE_QUEUE_BROADCASTER: '0'
SGLANG_DISABLE_TP_MEMORY_INBALANCE_CHECK: '1'
SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK: '0'
SGLANG_HEALTH_CHECK_TIMEOUT: '3600'
SGLANG_HEALTH_STARTING_OK: '1'
SGLANG_ENABLE_HEALTH_ENDPOINT_GENERATION: '0'
Expand All @@ -54,7 +54,7 @@ roles:
tensor-parallel-size: 4
data-parallel-size: 1
expert-parallel-size: 1
mamba-scheduler-strategy: no_buffer
mamba-radix-cache-strategy: no_buffer
mamba-ssm-dtype: bfloat16
mamba-track-interval: 2048
attention-backend: trtllm_mha
Expand All @@ -69,7 +69,8 @@ roles:
context-length: 9236
max-total-tokens: 128000
max-running-requests: 128
cuda-graph-max-bs: 4
cuda-graph-max-bs-decode: 4
cuda-graph-backend-prefill: disabled
chunked-prefill-size: 32768
max-prefill-tokens: 32768
scheduler-recv-interval: 1
Expand Down Expand Up @@ -106,7 +107,7 @@ roles:
SGLANG_DECODE_BOOTSTRAP_TIMEOUT: '1000'
SGLANG_MOONCAKE_CUSTOM_MEM_POOL: 'True'
SGLANG_USE_MESSAGE_QUEUE_BROADCASTER: '0'
SGLANG_DISABLE_TP_MEMORY_INBALANCE_CHECK: '1'
SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK: '0'
SGLANG_HEALTH_CHECK_TIMEOUT: '3600'
SGLANG_HEALTH_STARTING_OK: '1'
SGLANG_ENABLE_HEALTH_ENDPOINT_GENERATION: '0'
Expand All @@ -122,7 +123,7 @@ roles:
tensor-parallel-size: 4
data-parallel-size: 1
expert-parallel-size: 1
mamba-scheduler-strategy: no_buffer
mamba-radix-cache-strategy: no_buffer
mamba-ssm-dtype: bfloat16
mamba-track-interval: 128
attention-backend: trtllm_mha
Expand All @@ -142,7 +143,7 @@ roles:
max-total-tokens: 1500000
max-mamba-cache-size: 256
max-running-requests: 128
cuda-graph-max-bs: 256
cuda-graph-max-bs-decode: 256
chunked-prefill-size: 32768
max-prefill-tokens: 32768
scheduler-recv-interval: 10
Expand Down
13 changes: 7 additions & 6 deletions ...ks/multi_node/srt-slurm-recipes/qwen3.5/sglang/gb200-fp4-v2/8k1k/mtp/1p1d-tp4-tp4-c8.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ roles:
SGLANG_DISAGGREGATION_WAITING_TIMEOUT: '100000'
SGLANG_MOONCAKE_CUSTOM_MEM_POOL: 'True'
SGLANG_USE_MESSAGE_QUEUE_BROADCASTER: '0'
SGLANG_DISABLE_TP_MEMORY_INBALANCE_CHECK: '1'
SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK: '0'
SGLANG_HEALTH_CHECK_TIMEOUT: '3600'
SGLANG_HEALTH_STARTING_OK: '1'
SGLANG_ENABLE_HEALTH_ENDPOINT_GENERATION: '0'
Expand All @@ -54,7 +54,7 @@ roles:
tensor-parallel-size: 4
data-parallel-size: 1
expert-parallel-size: 1
mamba-scheduler-strategy: no_buffer
mamba-radix-cache-strategy: no_buffer
mamba-ssm-dtype: bfloat16
mamba-track-interval: 2048
attention-backend: trtllm_mha
Expand All @@ -69,7 +69,8 @@ roles:
context-length: 9236
max-total-tokens: 128000
max-running-requests: 128
cuda-graph-max-bs: 4
cuda-graph-max-bs-decode: 4
cuda-graph-backend-prefill: disabled
chunked-prefill-size: 32768
max-prefill-tokens: 32768
scheduler-recv-interval: 10
Expand Down Expand Up @@ -106,7 +107,7 @@ roles:
SGLANG_DECODE_BOOTSTRAP_TIMEOUT: '1000'
SGLANG_MOONCAKE_CUSTOM_MEM_POOL: 'True'
SGLANG_USE_MESSAGE_QUEUE_BROADCASTER: '0'
SGLANG_DISABLE_TP_MEMORY_INBALANCE_CHECK: '1'
SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK: '0'
SGLANG_HEALTH_CHECK_TIMEOUT: '3600'
SGLANG_HEALTH_STARTING_OK: '1'
SGLANG_ENABLE_HEALTH_ENDPOINT_GENERATION: '0'
Expand All @@ -122,7 +123,7 @@ roles:
tensor-parallel-size: 4
data-parallel-size: 1
expert-parallel-size: 1
mamba-scheduler-strategy: no_buffer
mamba-radix-cache-strategy: no_buffer
mamba-ssm-dtype: bfloat16
mamba-track-interval: 128
attention-backend: trtllm_mha
Expand All @@ -142,7 +143,7 @@ roles:
max-total-tokens: 1500000
max-mamba-cache-size: 256
max-running-requests: 128
cuda-graph-max-bs: 256
cuda-graph-max-bs-decode: 256
chunked-prefill-size: 32768
max-prefill-tokens: 32768
scheduler-recv-interval: 10
Expand Down
13 changes: 7 additions & 6 deletions .../multi_node/srt-slurm-recipes/qwen3.5/sglang/gb200-fp4-v2/8k1k/mtp/2p2d-tp2-tp2-c512.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ roles:
SGLANG_DISAGGREGATION_WAITING_TIMEOUT: '100000'
SGLANG_MOONCAKE_CUSTOM_MEM_POOL: 'True'
SGLANG_USE_MESSAGE_QUEUE_BROADCASTER: '0'
SGLANG_DISABLE_TP_MEMORY_INBALANCE_CHECK: '1'
SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK: '0'
SGLANG_HEALTH_CHECK_TIMEOUT: '3600'
SGLANG_HEALTH_STARTING_OK: '1'
SGLANG_ENABLE_HEALTH_ENDPOINT_GENERATION: '0'
Expand All @@ -58,7 +58,7 @@ roles:
tensor-parallel-size: 2
data-parallel-size: 1
expert-parallel-size: 1
mamba-scheduler-strategy: no_buffer
mamba-radix-cache-strategy: no_buffer
mamba-ssm-dtype: bfloat16
mamba-track-interval: 2048
attention-backend: trtllm_mha
Expand All @@ -72,7 +72,8 @@ roles:
context-length: 9236
max-total-tokens: 64000
max-running-requests: 64
cuda-graph-max-bs: 4
cuda-graph-max-bs-decode: 4
cuda-graph-backend-prefill: disabled
chunked-prefill-size: 16384
max-prefill-tokens: 16384
scheduler-recv-interval: 10
Expand Down Expand Up @@ -110,7 +111,7 @@ roles:
SGLANG_DECODE_BOOTSTRAP_TIMEOUT: '1000'
SGLANG_MOONCAKE_CUSTOM_MEM_POOL: 'True'
SGLANG_USE_MESSAGE_QUEUE_BROADCASTER: '0'
SGLANG_DISABLE_TP_MEMORY_INBALANCE_CHECK: '1'
SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK: '0'
SGLANG_HEALTH_CHECK_TIMEOUT: '3600'
SGLANG_HEALTH_STARTING_OK: '1'
SGLANG_ENABLE_HEALTH_ENDPOINT_GENERATION: '0'
Expand All @@ -126,7 +127,7 @@ roles:
tensor-parallel-size: 2
data-parallel-size: 1
expert-parallel-size: 1
mamba-scheduler-strategy: no_buffer
mamba-radix-cache-strategy: no_buffer
mamba-ssm-dtype: bfloat16
mamba-track-interval: 128
attention-backend: trtllm_mha
Expand All @@ -145,7 +146,7 @@ roles:
max-total-tokens: 750000
max-mamba-cache-size: 128
max-running-requests: 84
cuda-graph-max-bs: 88
cuda-graph-max-bs-decode: 88
chunked-prefill-size: 32768
max-prefill-tokens: 32768
scheduler-recv-interval: 10
Expand Down
13 changes: 7 additions & 6 deletions .../multi_node/srt-slurm-recipes/qwen3.5/sglang/gb200-fp4-v2/8k1k/mtp/2p3d-tp2-tp2-c768.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ roles:
SGLANG_DISAGGREGATION_WAITING_TIMEOUT: '100000'
SGLANG_MOONCAKE_CUSTOM_MEM_POOL: 'True'
SGLANG_USE_MESSAGE_QUEUE_BROADCASTER: '0'
SGLANG_DISABLE_TP_MEMORY_INBALANCE_CHECK: '1'
SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK: '0'
SGLANG_HEALTH_CHECK_TIMEOUT: '3600'
SGLANG_HEALTH_STARTING_OK: '1'
SGLANG_ENABLE_HEALTH_ENDPOINT_GENERATION: '0'
Expand All @@ -55,7 +55,7 @@ roles:
tensor-parallel-size: 2
data-parallel-size: 1
expert-parallel-size: 1
mamba-scheduler-strategy: no_buffer
mamba-radix-cache-strategy: no_buffer
mamba-ssm-dtype: bfloat16
mamba-track-interval: 2048
attention-backend: trtllm_mha
Expand All @@ -69,7 +69,8 @@ roles:
context-length: 9236
max-total-tokens: 64000
max-running-requests: 64
cuda-graph-max-bs: 4
cuda-graph-max-bs-decode: 4
cuda-graph-backend-prefill: disabled
chunked-prefill-size: 16384
max-prefill-tokens: 16384
scheduler-recv-interval: 10
Expand Down Expand Up @@ -106,7 +107,7 @@ roles:
SGLANG_DECODE_BOOTSTRAP_TIMEOUT: '1000'
SGLANG_MOONCAKE_CUSTOM_MEM_POOL: 'True'
SGLANG_USE_MESSAGE_QUEUE_BROADCASTER: '0'
SGLANG_DISABLE_TP_MEMORY_INBALANCE_CHECK: '1'
SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK: '0'
SGLANG_HEALTH_CHECK_TIMEOUT: '3600'
SGLANG_HEALTH_STARTING_OK: '1'
SGLANG_ENABLE_HEALTH_ENDPOINT_GENERATION: '0'
Expand All @@ -122,7 +123,7 @@ roles:
tensor-parallel-size: 2
data-parallel-size: 1
expert-parallel-size: 1
mamba-scheduler-strategy: no_buffer
mamba-radix-cache-strategy: no_buffer
mamba-ssm-dtype: bfloat16
mamba-track-interval: 128
attention-backend: trtllm_mha
Expand All @@ -141,7 +142,7 @@ roles:
max-total-tokens: 750000
max-mamba-cache-size: 128
max-running-requests: 86
cuda-graph-max-bs: 88
cuda-graph-max-bs-decode: 88
chunked-prefill-size: 32768
max-prefill-tokens: 32768
scheduler-recv-interval: 10
Expand Down
13 changes: 7 additions & 6 deletions .../multi_node/srt-slurm-recipes/qwen3.5/sglang/gb200-fp4-v2/8k1k/mtp/3p5d-tp4-tp4-c128.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ roles:
SGLANG_DISAGGREGATION_WAITING_TIMEOUT: '100000'
SGLANG_MOONCAKE_CUSTOM_MEM_POOL: 'True'
SGLANG_USE_MESSAGE_QUEUE_BROADCASTER: '0'
SGLANG_DISABLE_TP_MEMORY_INBALANCE_CHECK: '1'
SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK: '0'
SGLANG_HEALTH_CHECK_TIMEOUT: '3600'
SGLANG_HEALTH_STARTING_OK: '1'
SGLANG_ENABLE_HEALTH_ENDPOINT_GENERATION: '0'
Expand All @@ -54,7 +54,7 @@ roles:
tensor-parallel-size: 4
data-parallel-size: 1
expert-parallel-size: 1
mamba-scheduler-strategy: no_buffer
mamba-radix-cache-strategy: no_buffer
mamba-ssm-dtype: bfloat16
mamba-track-interval: 2048
attention-backend: trtllm_mha
Expand All @@ -69,7 +69,8 @@ roles:
context-length: 9236
max-total-tokens: 128000
max-running-requests: 128
cuda-graph-max-bs: 4
cuda-graph-max-bs-decode: 4
cuda-graph-backend-prefill: disabled
chunked-prefill-size: 32768
max-prefill-tokens: 32768
scheduler-recv-interval: 10
Expand Down Expand Up @@ -106,7 +107,7 @@ roles:
SGLANG_DECODE_BOOTSTRAP_TIMEOUT: '1000'
SGLANG_MOONCAKE_CUSTOM_MEM_POOL: 'True'
SGLANG_USE_MESSAGE_QUEUE_BROADCASTER: '0'
SGLANG_DISABLE_TP_MEMORY_INBALANCE_CHECK: '1'
SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK: '0'
SGLANG_HEALTH_CHECK_TIMEOUT: '3600'
SGLANG_HEALTH_STARTING_OK: '1'
SGLANG_ENABLE_HEALTH_ENDPOINT_GENERATION: '0'
Expand All @@ -122,7 +123,7 @@ roles:
tensor-parallel-size: 4
data-parallel-size: 1
expert-parallel-size: 1
mamba-scheduler-strategy: no_buffer
mamba-radix-cache-strategy: no_buffer
mamba-ssm-dtype: bfloat16
mamba-track-interval: 128
attention-backend: trtllm_mha
Expand All @@ -142,7 +143,7 @@ roles:
max-total-tokens: 1500000
max-mamba-cache-size: 256
max-running-requests: 128
cuda-graph-max-bs: 256
cuda-graph-max-bs-decode: 256
chunked-prefill-size: 32768
max-prefill-tokens: 32768
scheduler-recv-interval: 10
Expand Down
13 changes: 7 additions & 6 deletions ...ti_node/srt-slurm-recipes/qwen3.5/sglang/gb200-fp4-v2/8k1k/mtp/7p1d-dep4-dep16-c5120.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ roles:
SGLANG_DISAGGREGATION_WAITING_TIMEOUT: '100000'
SGLANG_MOONCAKE_CUSTOM_MEM_POOL: 'True'
SGLANG_USE_MESSAGE_QUEUE_BROADCASTER: '0'
SGLANG_DISABLE_TP_MEMORY_INBALANCE_CHECK: '1'
SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK: '0'
SGLANG_HEALTH_CHECK_TIMEOUT: '3600'
SGLANG_HEALTH_STARTING_OK: '1'
SGLANG_ENABLE_HEALTH_ENDPOINT_GENERATION: '0'
Expand All @@ -58,7 +58,7 @@ roles:
expert-parallel-size: 4
enable-dp-attention: true
enable-dp-lm-head: true
mamba-scheduler-strategy: no_buffer
mamba-radix-cache-strategy: no_buffer
mamba-ssm-dtype: bfloat16
mamba-track-interval: 2048
attention-backend: trtllm_mha
Expand All @@ -72,7 +72,8 @@ roles:
context-length: 9236
max-total-tokens: 128000
max-running-requests: 128
cuda-graph-max-bs: 4
cuda-graph-max-bs-decode: 4
cuda-graph-backend-prefill: disabled
chunked-prefill-size: 65536
max-prefill-tokens: 65536
scheduler-recv-interval: 1
Expand Down Expand Up @@ -111,7 +112,7 @@ roles:
SGLANG_DECODE_BOOTSTRAP_TIMEOUT: '1000'
SGLANG_MOONCAKE_CUSTOM_MEM_POOL: 'True'
SGLANG_USE_MESSAGE_QUEUE_BROADCASTER: '0'
SGLANG_DISABLE_TP_MEMORY_INBALANCE_CHECK: '1'
SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK: '0'
SGLANG_HEALTH_CHECK_TIMEOUT: '3600'
SGLANG_HEALTH_STARTING_OK: '1'
SGLANG_ENABLE_HEALTH_ENDPOINT_GENERATION: '0'
Expand All @@ -129,7 +130,7 @@ roles:
expert-parallel-size: 16
enable-dp-attention: true
enable-dp-lm-head: true
mamba-scheduler-strategy: no_buffer
mamba-radix-cache-strategy: no_buffer
mamba-ssm-dtype: bfloat16
mamba-track-interval: 128
attention-backend: trtllm_mha
Expand All @@ -147,7 +148,7 @@ roles:
disaggregation-mode: decode
disaggregation-transfer-backend: mooncake
disable-radix-cache: true
cuda-graph-max-bs: 256
cuda-graph-max-bs-decode: 256
mem-fraction-static: 0.8
context-length: 9236
max-total-tokens: 2200000
Expand Down
Loading
Loading