[chenfeiz0326]

Description

nvbugs/6958731: the three MiniMax-M3 AgentX lanes in GB300 post-merge perf-sanity (aggr_upload-gb300_minimax_m3_fp4_agentx_grace_blackwell-*) have failed on every run since they were added. trtllm-serve answered every POST /v1/chat/completions with HTTP 400, so the readiness probe never completed one token and each lane timed out after 900 s. With #20749 in place, the probe reports the server's reason:

HTTP 400: {"object":"error","message":"No chat template found for the given tokenizer and tools.","type":"BadRequestError","param":null,"code":400}

Root cause.

• resolve_hf_chat_template returned processor.chat_template for every request without tools whenever a processor was loaded, even when that attribute was None.
• apply_chat_template then raised ValueError("No chat template found for the given tokenizer and tools."), and openai_chat turns that into HTTP 400 without logging a traceback.
• trtllm-serve loads AutoProcessor for every checkpoint. For MiniMax-M3-NVFP4 that is the checkpoint's remote-code MiniMaxVLProcessor, whose chat_template is None. The tokenizer, and the processor's own tokenizer, carry the checkpoint's chat_template.jinja (11,752 characters).
• Requests with tools take the tokenizer path and were unaffected. Every request without tools failed, including the readiness probe's minimal {"messages":[{"role":"user","content":"hi"}],"max_tokens":1}.

Fix. Use the processor's template only when it has one; otherwise fall through to the tokenizer's template. vLLM resolves the same way: resolve_chat_template in vllm/renderers/hf.py takes the processor's template only when it is not None, then falls back to the tokenizer.

Test Coverage

• New TestProcessorWithoutChatTemplate in tests/unittest/inputs/test_chat_template_dispatch.py, which is cpu_only and runs in l0_cpu through the unittest/inputs entry:
  • A processor without a template falls back to the tokenizer's template. On main this raises the same ValueError the server returns.
  • A processor template is still preferred over the tokenizer's.
  • Run in the CI container on GB300 (build 3150 wheel, pytest.ini with pytest-threadleak): the new fallback test fails against the installed module and passes with this change. With the change, test_chat_template_dispatch.py and test_chat_template_kwargs_guard.py pass (78 tests).
• GB300 reproduction on oci-jhb, using build 3150 (b833b7c9) and CI's config for the tp4 con1 lane:
  • On a freshly started server, /v1/chat/completions returned the 400 above, while /v1/completions returned 200 with a generated token.
  • The result was the same after the AgentX client's own pip install, so that install is not involved.
  • In the same container, AutoProcessor gave MiniMaxVLProcessor with chat_template=None, and the tokenizer held the 11,752-character template.
  • The installed apply_chat_template raised the ValueError. With the fixed resolver it rendered the MiniMax-M3 prompt, identical to tokenizer.apply_chat_template.
• GB300 end-to-end with this fix, on oci-jhb: build 3150 wheel with this change overlaid, CI's test_e2e[aggr-gb300_minimax_m3_fp4_agentx_grace_blackwell-minimax_m3_fp4_tp4_eagle3_mtp7_con1_agentx], unmodified AgentX client. The lane passed (1 passed, 1 h 20 min).
  • The readiness probe got its token: 1-token completion OK.
  • AgentX reported health OK: 267 requests over 3614.2s, submission_valid=true. One empty-content response was within the 0.5% tolerance.
  • The server returned 273 chat completions with HTTP 200 and none with 4xx/5xx.
  • Results: 0.07 req/s, 141.13 output tok/s, median TTFT 967 ms, mean TPOT 1.95 ms.

PR Checklist

Please review the following before submitting your PR:

• PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

• PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

• Test cases are provided for new code paths (see test instructions)

• If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

• Any new dependencies have been scanned for license and vulnerabilities

• CODEOWNERS updated if ownership changes

• Documentation updated as needed

• Update tava architecture diagram if there is a significant design change in PR.

• The reviewers assigned automatically/manually are appropriate for the PR.

• Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

🤖 Generated with Claude Code

Dev Engineer Review

resolve_hf_chat_template uses a non-None processor chat template when applicable. If the processor template is missing or None, resolution falls back to the tokenizer’s template. No current review findings were supplied.

QA Engineer Review

tests/unittest/inputs/test_chat_template_dispatch.py adds TestProcessorWithoutChatTemplate tests for tokenizer fallback and processor-template precedence. The tests cover the reported missing-template behavior. No integration test-list change appears applicable to this unit test. The PR description reports 78 passing tests and a passing GB300 MiniMax-M3 end-to-end run. Coverage verdict: sufficient for the changed resolution behavior.

Per-File QA Perspective

• tensorrt_llm/inputs/utils.py: Verify that a present processor template takes precedence and that a missing or None template falls back to tokenizer lookup. Confirm explicit-template and tool-aware behavior remains unchanged.
• tests/unittest/inputs/test_chat_template_dispatch.py: Covers processor-template precedence and tokenizer fallback. The unit test does not require an integration CI or manual-QA list entry.

[chenfeiz0326]

/bot run --disable-fail-fast

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

• Source mode: Auto
• Repository: NVIDIA/TensorRT-LLM
• Branch: main
• File: .coderabbit.yaml
• Commit: 12af626
• Layers: Repository YAML
• Review profile: CHILL
• Plan: Enterprise
• Run ID: 296a9c2f-9ea6-4d59-81c5-45928d0f8228

📥 Commits

Reviewing files that changed from the base of the PR and between 8f49021 and 58112fb.

📒 Files selected for processing: 1

• tests/unittest/inputs/test_chat_template_dispatch.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 10 remain after this review.

Walkthrough

The chat-template resolver now falls back to tokenizer lookup when the processor does not provide a template. Added tests check tokenizer fallback and processor-template precedence.

Changes

Chat template resolution

Layer / File(s)	Summary
Template selection and validation tensorrt_llm/inputs/utils.py, tests/unittest/inputs/test_chat_template_dispatch.py	When tools are absent, the resolver uses the processor’s template if it is non-None. Otherwise, it falls through to tokenizer template lookup. Tests cover tokenizer fallback and processor-template precedence.

Priority: ➖ Normal

Estimated code review effort: 2 (Simple) | ~8 minutes

Change: Bug fix

Possibly related PRs

• NVIDIA/TensorRT-LLM#20749: Addresses readiness rejection reporting for the same MiniMax-M3 AgentX failure context.

Merge Risk: ⚪ Minimal · up to 58112

The reported chat-template failure is addressed, with focused tests covering fallback and precedence.

Pre-merge checks: 4 · 1

Failed checks: 1 warning

Check name	Status	Explanation	Resolution
Docstring Coverage		Docstring coverage is 12.50% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 8 functions across 2 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

Passed checks: 4 passed

Check name	Status	Explanation
Title check		The title follows the required NVBugs and type format and clearly describes the primary fix: falling back to the tokenizer chat template when the processor has none.
Description check		The description is complete and relevant. It explains the failure, root cause, fix, test coverage, end-to-end validation, and checklist status.
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}

[tensorrt-cicd]

PR_Github #80097 [ run ] triggered by Bot. Commit: 8f49021 Link to invocation

[chenfeiz0326]

GB300 end-to-end result with this change: the MiniMax-M3 tp4 con1 AgentX lane passed on oci-jhb. I ran the build 3150 wheel with this fix overlaid and the unmodified client, under CI's own test_e2e config.

• The readiness probe now gets its token (1-token completion OK). Without the fix, the same probe got HTTP 400: No chat template found for the given tokenizer and tools.
• AgentX reported health OK: 267 requests over 3614.2s, submission_valid=true.
• The server returned 273 chat completions with HTTP 200 and none with 4xx/5xx.
• pytest reported 1 passed (1 h 20 min).

[JunyiXu-nv]

Approve with a nit in test.

[github-actions]

Automatically added "ci: full pre-merge approved" because this PR has satisfied the required GitHub review approvals. Unresolved review conversations and other required checks remain independent merge requirements.

[chenfeiz0326]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #80134 [ run ] triggered by Bot. Commit: 58112fb Link to invocation

[tensorrt-cicd]

PR_Github #80097 [ run ] completed with state ABORTED. Commit: 8f49021

Link to invocation

[chenfeiz0326]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #80179 [ run ] triggered by Bot. Commit: 58112fb Link to invocation

[tensorrt-cicd]

PR_Github #80134 [ run ] completed with state ABORTED. Commit: 58112fb

Link to invocation

[chenfeiz0326]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #80183 [ run ] triggered by Bot. Commit: 58112fb Link to invocation

[tensorrt-cicd]

PR_Github #80179 [ run ] completed with state ABORTED. Commit: 58112fb

Link to invocation

[brnguyen2]

The resolver change matches how the native-template guard in apply_chat_template already treats a processor whose chat_template is None, and the two new tests cover both resolution orders. I have no changes to request.

[tensorrt-cicd]

PR_Github #80183 [ run ] completed with state SUCCESS. Commit: 58112fb
/LLM/main/L0_MergeRequest_PR pipeline #65990 completed with status: 'SUCCESS'

CI Report

Link to invocation