Skip to content

Add onnxruntime-gpu (ONNX-on-GPU) install steps to TensorRT doc - #12

Merged
kouya-marino merged 2 commits into
mainfrom
docs/onnxruntime-gpu-steps
May 31, 2026
Merged

kouya-marino merged 2 commits into
mainfrom
docs/onnxruntime-gpu-steps

Conversation

@kouya-marino

Copy link
Copy Markdown
Owner

Adds a section to docs/TensorRT.md on running the plain ONNX Deploy path on GPU via onnxruntime-gpu (no TensorRT needed):

  • Version matching table — onnxruntime-gpu 1.17.x → CUDA 11.8 / cuDNN 8; 1.18+ → CUDA 12 / cuDNN 9.
  • Install commands for the CUDA 11.8 (e.g. PyTorch cu118) and CUDA 12.x cases, with PATH/cuDNN notes and the "only one of onnxruntime/onnxruntime-gpu" caveat.
  • Verification (get_available_providers() → CUDAExecutionProvider; Deploy side panel shows the provider).

Directly addresses the earlier "ONNX is slower than .pth" finding (it was falling back to CPU because onnxruntime-gpu didn't match the installed CUDA). Troubleshooting renumbered to §9. Docs-only.

prashantrawat11 added 2 commits May 31, 2026 14:14
New section in docs/TensorRT.md covering how to run the ONNX Deploy path on GPU via onnxruntime-gpu: CUDA/cuDNN version matching (1.17.x->CUDA 11.8/cuDNN 8; 1.18+->CUDA 12/cuDNN 9), CUDA 11.8 vs 12.x install commands, PATH notes, and verification. Troubleshooting renumbered to section 9.
Make explicit that onnxruntime-gpu does not bundle CUDA/cuDNN (it loads them from PATH), that PyTorch's bundled libs don't satisfy it, and that a working setup is the right build PLUS matching CUDA/cuDNN on PATH. Reframe as Route A (match CUDA 12, simplest with a recent onnxruntime) vs Route B (match CUDA 11.8 + cuDNN 8), and clarify the cuDNN major pairs with the onnxruntime build.
@kouya-marino
kouya-marino merged commit a0addad into main May 31, 2026
4 checks passed
@kouya-marino
kouya-marino deleted the docs/onnxruntime-gpu-steps branch May 31, 2026 05:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant