Pinned Loading

1. sglang sglang Public

   Forked from sgl-project/sglang

   SGLang is a fast serving framework for large language models and vision language models.

   Python 1

2. vllm vllm Public

   Forked from vllm-project/vllm

   A high-throughput and memory-efficient inference and serving engine for LLMs

   Python

3. nccl nccl Public

   Forked from NVIDIA/nccl

   Optimized primitives for collective multi-GPU communication

   C++

4. Mooncake Mooncake Public

   Forked from kvcache-ai/Mooncake

   Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

   C++

5. TensorRT-Model-Optimizer TensorRT-Model-Optimizer Public

   Forked from NVIDIA/Model-Optimizer

   A unified library of state-of-the-art model optimization techniques like quantization, pruning, distillation, speculative decoding, etc. It compresses deep learning models for downstream deployment…

   Python

6. transformers transformers Public

   Forked from huggingface/transformers

   🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

   Python