[yufeiwu-nv]

Description

Add core performance coverage for GLM-5.3 FP8 (TP8/EP8) and GLM-5.3-Flash FP8 + MTP3 (TP4/EP4): checkpoint aliases, serving recipes, and four cases covering concurrency 1 and 64 with ISL/OSL 1024/1024.

The cases are integrated directly into tests/integration/test_lists/qa/llm_perf_core.yml, reusing its existing hardware conditions:

• Flash uses condition 6: at least four GPUs, compute capability 10.0–10.7, including the existing VR200 coverage.
• Full GLM-5.3 uses condition 7: at least eight GPUs, compute capability 10.0–10.3.

No hardware conditions are added or changed. The standalone onboarding list is removed.

Full GLM-5.3 FP8 cases allow 1800 seconds for checkpoint loading and cold executor warmup, instead of the default 600-second server-startup timeout. Flash text-only cases set disable_mm_encoder=True; the MTP alias follows the speculative-decoding EOS policy, so the output length is a cap and performance reporting uses actual generated tokens.

Test Coverage

• Repository pre-commit and commit-message checks passed, including YAML validation, test-list AST validation, and DCO.
• Verified that the original 13 hardware conditions and their previous cases are preserved, and that the four case IDs and timeout annotations appear exactly once in the intended groups.
• Local configuration checks passed for case parsing/round-trip, TP/EP/GPU counts, recipe selection, MTP/EOS behavior, and unchanged recipes for other models.
• The final startup-timeout and core-list source changes were checked locally; this update does not include a new GPU run.

PR Checklist

• Changes are limited to performance harness configuration and the core test list. No public API, dependency, or ownership changes.

Dev Engineer Review

The changes add aliases for GLM-5.3 FP8 and GLM-5.3-Flash FP8 MTP3 checkpoints. The performance recipes configure the base model for TP8/EP8 with FP8 KV cache and DEEPGEMM MoE, and the Flash model for TP4/EP4 with text-only encoder settings and TRTLLM MoE. The Flash MTP recipe enables draft length 3.

The test configuration adds GLM-5.3-Flash MTP to speculative-decoding handling, and sets an 1,800-second server startup timeout for full GLM-5.3 FP8. The QA list places the Flash cases in the four-GPU condition and the full-model cases in the eight-GPU condition. All four entries use TIMEOUT(120). No new GPU run is reported, so runtime performance remains unverified.

QA Engineer Review

tests/integration/test_lists/qa/llm_perf_core.yml adds four manual QA performance entries: concurrency 1 and 64 for each model variant, with ISL/OSL 1024/1024. No test-function changes are reported. The supplied evidence does not include a GPU run; coverage verdict: needs follow-up.

Per-File QA Perspective

• tests/integration/defs/perf/_model_paths.py: Adds checkpoint aliases for the full and Flash configurations. Verify that each alias resolves to the intended checkpoint.
• tests/integration/defs/perf/pytorch_model_config.py: Adds serving recipes, including FP8 KV cache for the full model, text-only encoder settings for Flash, and MTP3 settings for the Flash MTP alias. Verify recipe selection and runtime behavior.
• tests/integration/defs/perf/test_perf.py: Adds the Flash MTP alias to speculative-decoding handling and increases the full GLM-5.3 FP8 startup timeout to 1,800 seconds. Verify EOS handling and cold-start behavior.
• tests/integration/test_lists/qa/llm_perf_core.yml: Adds two Flash MTP cases under the four-GPU hardware condition and two full-model cases under the eight-GPU condition. Verify both concurrency settings and performance results on applicable hardware.

[yufeiwu-nv]

/bot run

[coderabbitai]

No actionable comments were generated in the recent review.

Recent review info

Run configuration

• Source mode: Auto
• Repository: NVIDIA/TensorRT-LLM
• Branch: main
• File: .coderabbit.yaml
• Commit: 0cd0a18
• Layers: Repository YAML
• Review profile: CHILL
• Plan: Enterprise
• Run ID: 1d5906da-171a-46e1-b872-1e465a7f14dd

Commits

Reviewing files that changed from the base of the PR and between 0cd0a18 and 0c2ba76.

Files selected for processing: 4

• tests/integration/defs/perf/_model_paths.py
• tests/integration/defs/perf/pytorch_model_config.py
• tests/integration/defs/perf/test_perf.py
• tests/integration/test_lists/qa/llm_perf_core.yml

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

Walkthrough

The change adds GLM-5.3 FP8 and GLM-5.3-Flash FP8 MTP model paths and PyTorch serving configurations. It adds speculative-decoding support and startup timeout handling, plus streaming performance cases for 4-GPU and 8-GPU conditions.

Changes

GLM-5.3 Performance Coverage

Layer / File(s)	Summary
Model paths and serving configuration tests/integration/defs/perf/_model_paths.py, tests/integration/defs/perf/pytorch_model_config.py, tests/integration/defs/perf/test_perf.py	Adds model paths and PyTorch serving configurations for GLM-5.3 FP8 and GLM-5.3-Flash FP8. The Flash MTP configuration enables draft decoding, the MTP model joins the speculative-decoding set, and GLM-5.3 FP8 uses a 1,800-second server startup timeout.
GPU performance cases tests/integration/test_lists/qa/llm_perf_core.yml	Adds two 4-GPU streaming cases for GLM-5.3-Flash FP8 MTP and two 8-GPU streaming cases for GLM-5.3 FP8 without MTP. Each case specifies a 120-second timeout.

Priority: Normal

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Other

Suggested labels: api-compatible

Merge Risk: Minimal · up to 0c2ba

The added performance coverage has no established merge-blocking issue and can proceed to normal GPU validation.

Pre-merge checks: 4 · 1

Failed checks: 1 warning

Check name	Status	Explanation	Resolution
Docstring Coverage		Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documenta…	Write docstrings for the functions missing them to satisfy the coverage threshold.

Passed checks: 4 passed

Check name	Status	Explanation
Title check		The title follows the required [None][type] format and clearly identifies the addition of GLM-5.3 core performance coverage.
Description check		The description includes the required Description, Test Coverage, and PR Checklist sections. It explains the changes, affected configurations, validation performed, and scope limitations. The content …
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.

Full details: Description check

Explanation

The description includes the required Description, Test Coverage, and PR Checklist sections. It explains the changes, affected configurations, validation performed, and scope limitations. The content is sufficiently complete despite not checking the final checklist confirmation box.

Full details: Docstring Coverage

Explanation

Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 2 functions across 3 files. (1 skipped: 1 unsupported.)

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

Finishing Touches

🧪 Generate unit tests (beta)

• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}

[tensorrt-cicd]

PR_Github #81170 [ run ] triggered by Bot. Commit: 0c2ba76 Link to invocation

[tensorrt-cicd]

PR_Github #81170 [ run ] completed with state SUCCESS. Commit: 0c2ba76
/LLM/main/L0_MergeRequest_PR pipeline #66866 completed with status: 'UNSTABLE'

CI Report

⚠️ Multi-GPU Label Required:
Multi-GPU tests require the ci: full pre-merge approved label on this PR. Either:

• Wait for the PR to be fully approved — the label is added automatically once approval is complete. Having unresolved open comments is fine, or
• If needed, ask a member of NVIDIA/trt-llm-ci-approvers to add the label manually.
  Then re-trigger CI with the same bot command (no rebase needed).

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation