Skip to content

Pinned Loading

  1. vllm vllm Public

    A high-throughput and memory-efficient inference and serving engine for LLMs

    Python 59.4k 10.5k

  2. llm-compressor llm-compressor Public

    Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

    Python 2k 247

  3. recipes recipes Public

    Common recipes to run vLLM

    Jupyter Notebook 149 50

Repositories

Showing 10 of 24 repositories