docker: remove the base image's apt cuDNN so the pip one is what loads

[yueming-yuan]

Follow-up to #1808. That PR raised the pip cuDNN pin to 9.22.0.52, which is
necessary but not sufficient — on current main the failure it targeted still
reproduces.

The pin was inert

The base image carries cuDNN as apt packages (libcudnn9-cuda-13 9.13.0.50-1,
plus -dev- and -headers-) in /usr/lib/x86_64-linux-gnu, and ldconfig
resolves the sonames there ahead of the pip copy in
.../dist-packages/nvidia/cudnn/lib. The loader then interleaves the two:

OSError: /lib/x86_64-linux-gnu/libcudnn_heuristic.so.9: undefined symbol:
         _ZTVN5cudnn7backend12OperationSetE, version libcudnn_graph.so.9

libcudnn_heuristic.so.9 comes from the apt build while libcudnn_graph.so.9
comes from the pip build. Depending on load order transformer_engine dies either
at import with that undefined symbol, or later inside fused attention with the
CUDNN_ATTR_OPERATION_RESHAPE_MODE CUDNN_STATUS_BAD_PARAM that #1808 set out
to fix. Either way, moving the pip pin alone changes nothing — the pip build is
not what gets loaded.

The apt packages are held (hi in dpkg -l), so the removal needs
--allow-change-held-packages. Without that flag apt reports the plan and skips,
leaving everything in place.

Measurements

On radixark/miles:pr-1795@sha256:792974ed (H200), 8 te.DotProductAttention
forward+backward configs — d=64/128 × MHA(8/8)/GQA(8/2) × sbhd/bshd, bf16, causal,
NVTE_FUSED_ATTN=1:

apt cuDNN 9.13	pip cuDNN	result
present	9.16.0.29	torch refuses to init cuDNN (compiled 9.19.0, found 9.16.0)
present	9.22.0.52	8/8 fail
removed	9.19.0.56	8/8 fail — BAD_PARAM at fused_attn_f16_arbitrary_seqlen.cu:934
removed	9.22.0.52	8/8 pass

Row 3 is why the pin stays at 9.22.0.52 rather than being dropped: with the
conflict gone, torch's own bundled 9.19.0.56 still fails, so #1808's version
floor is real. Both halves are required.

Removing the apt packages breaks nothing else — the libc10.so / libtorch_*.so
"not found" that ldd reports for the apex extensions is pre-existing and
identical before and after, and holoscan-cuda-13 was not pulled in by the
removal.

Scope

ENABLE_CUDA_13=1 is the default and what CI builds for both architectures, so
the cu13 path is what the measurements above cover. The cu12 branch is written
symmetrically; libcudnn9-cuda-12, libcudnn9-dev-cuda-12 and
libcudnn9-headers-cuda-12 all exist in the NVIDIA repo (9.24/9.25), but the
cu12-x86 variant was not built as part of this work.

Fixes the two LoRA tests that fail on #1795
(test_lora_qwen2.5_0.5B, test_qwen3_5_35b_a3b_lora_ci); test_dumper's
timeout is plausibly the same cause and is worth re-checking once this lands.

[gemini-code-assist]

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.