Package request: TensorRT-LLM

[ehfd]

Package name

TensorRT-LLM

Package version

Newest

Package website

https://docs.nvidia.com/tensorrt-llm/index.html

https://github.com/NVIDIA/TensorRT-LLM

Package availability

https://pypi.org/project/tensorrt-llm/

https://github.com/NVIDIA/TensorRT-LLM/releases

Additional comments

This package is a key interface for large language model inference on NVIDIA GPUs, and benefits the inference of various language and multimodal models including Llama and Stable Diffusion.

This depends on TensorRT (#25661).

Package is not available

• The package is not available on conda-forge.

No previous issues or open PRs

• No previous issue exists and no PR has been opened.