[TRTLLM-14477][refactor] group multimodal request data and encoder state

[yechank-nvidia]

Consolidated into #20391. Follow that PR for the combined implementation and validation.

Depends on #20391 (parent 75761fc826). Review only PR2.

The parent branch is on my fork, so this draft targets upstream main and currently includes the parent changes. The comparison above contains only this follow-up.

Description

Multimodal request data, item descriptions, encoder readiness and stream events are stored in separate places. Group them under LlmRequest.py_multimodal_params, while keeping selected encoder inputs and LLM chunks as separate Params that do not own cache references.

LlmRequest.py_multimodal_params
  multimodal_data
    item_metadata: MultimodalItemMetadata
      item_refs / encoder costs / output rows / input_partitions
  encoder_state / encoder_event
  multimodal_runtime = None

• Generalize item metadata beyond scheduling. Unknown encoder costs or output rows stay None; model capabilities and configuration still control scheduling. Normalize legacy order at input boundaries and keep one canonical item record in the existing payload key.
• Store the existing MultimodalItemPartition records under item_metadata.input_partitions. Qwen supplies packed vision partitions, Gemma4 supplies image/audio fields, and Nemotron supplies standalone audio clip groups. Nemotron temporal vision, EVS and embedded video audio keep their existing model handling.
• Move the existing pure encoder state into the data module and preserve its runtime imports. Legacy request fields alias the same Params. Data replacement preserves state; children copy the aggregate before admission. Cleanup releases every reference, including duplicate keys, and clears events.
• Preserve the three-field encoder metadata tuple as a producer/import adapter. Reuse the existing IPC/rank payload, C++ boundary, EncoderGroup, TensorLRUCache and runtime views. No serving/configuration changes or new scheduling capability.

Self-review found that a legacy caller adding another modality after normalization could build cache keys from incomplete item metadata. Keep that request on the existing uncacheable fallback; CUDA prefetch regression tests pass.

Validation

Revalidated after syncing both branches with main (3ac4e61255).

• CPU regression and API stability: 601 passed, 2 subtests; one CUDA case skipped and one deselected.
• CPU runtime, request handoff, no-KV runner and Nemotron preprocessing/audio checks: 614 passed, one CUDA case skipped.
• H200 model tests: Qwen3-VL image/video, deepstack, graph, chunked prefill and KV reuse (1 test + 7 subtests); Gemma4 vision pipeline, batched tower and multimodal E2E (3 tests). These use reduced/random weights and real processors.
• H200 cross-iteration prefetch: 16 passed, including cache hit/miss/partial-hit, failure and stream/event handling. CUDA partition selection: 63 cases. GPU-only handoff, fusion and device-grid checks: 3 passed.
• 12 real processor selection comparisons: Qwen3-VL image/video and Gemma4 audio preserve tensor values, shapes and dtypes for full, contiguous, reordered and single-item inputs.
• Real Qwen3-VL-2B-Instruct and Nemotron-Nano-12B-v2-VL-BF16 weights: image batch, repeated image and video generation. All 8 requests match the PR1 parent token ids and generation logits exactly.
• Pre-commit and DCO checks passed.

CPU input-preparation microbenchmark: PR1 dae1bb3a07 -> PR2 63b62770b6, measured before the final main sync. Eight items per request, three selected; median of nine samples, twenty repetitions per sample. Request counts below describe one preparation batch, not serving concurrency.

Requests prepared	Selection	PR1	PR2	Time reduction
64	Contiguous	0.838 ms	0.785 ms	6.3%
64	Reordered	1.049 ms	1.023 ms	2.4%
128	Contiguous	1.721 ms	1.615 ms	6.2%
128	Reordered	2.223 ms	2.187 ms	1.6%

Added Python partition metadata is about 0.5 KiB per selected view; contiguous tensor selections remain views. These measurements cover CPU input preparation; serving TTFT and throughput remain unmeasured.

Full Gemma4-26B initialization did not finish in this container on the parent branch; full-weight Gemma4 parity is not claimed. Multi-GPU and AIPerf serving checks remain for the shared batching follow-up.

PR Checklist

• Please check this after reviewing the above items as appropriate for this PR.