A high-throughput and memory-efficient inference and serving engine for LLMs

Community maintained hardware plugin for vLLM on Spyre

vLLM’s reference system for K8S-native cluster-wide deployment with community-driven performance optimization

This repo hosts code for vLLM CI & Performance Benchmark infrastructure.

Community maintained hardware plugin for vLLM on Ascend

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

The vLLM XPU kernels for Intel GPU

Community maintained hardware plugin for vLLM on Intel Gaudi

Cost-efficient and pluggable Infrastructure components for GenAI inference

Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs