Neural Magic

All

78 repositories

vllm
Public
A high-throughput and memory-efficient inference and serving engine for LLMs
Python
•
Apache License 2.0
•9.3k•13•0•7•Updated Aug 12, 2025Aug 12, 2025
compressed-tensors
Public
A safetensors extension to efficiently store sparse quantized tensors on disk
Python
•
Apache License 2.0
•19•143•6•21•Updated Aug 12, 2025Aug 12, 2025
speculators
Public
Python
•
Apache License 2.0
•3•24•3•8•Updated Aug 11, 2025Aug 11, 2025
research
Public
Repository to enable research flows
Python
•0•1•0•3•Updated Aug 11, 2025Aug 11, 2025
model-validation-configs
Public
0•0•0•1•Updated Aug 11, 2025Aug 11, 2025
axolotl
Public
Go ahead and axolotl questions
Python
•
Apache License 2.0
•1.1k•0•0•5•Updated Aug 10, 2025Aug 10, 2025
nm-actions
Public
Neural Magic GHA
Python
•
Apache License 2.0
•0•0•0•4•Updated Aug 7, 2025Aug 7, 2025
lmms-eval
Public
Accelerating the development of large multimodal models (LMMs) with one-click evaluation module - lmms-eval.
Python
•
Other
•356•0•0•9•Updated Aug 6, 2025Aug 6, 2025
DeepEP
Public
DeepEP: an efficient expert-parallel communication library
Cuda
•
MIT License
•891•0•0•0•Updated Jul 28, 2025Jul 28, 2025
flashinfer
Public
FlashInfer: Kernel Library for LLM Serving
Cuda
•
Apache License 2.0
•427•0•0•0•Updated Jul 18, 2025Jul 18, 2025
DeepGEMM
Public
DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling
Python
•
MIT License
•671•0•0•0•Updated Jul 18, 2025Jul 18, 2025
arena-hard-auto
Public
Arena-Hard-Auto: An automatic LLM benchmark.
Python
•
Apache License 2.0
•116•0•0•1•Updated Jul 16, 2025Jul 16, 2025
pplx-kernels
Public
Perplexity GPU Kernels
C++
•
MIT License
•51•0•0•0•Updated Jul 15, 2025Jul 15, 2025
collective_op_benchmarks
Public
Python
•0•0•0•0•Updated Jul 11, 2025Jul 11, 2025
LMCache
Public
Redis for LLMs
Python
•
Apache License 2.0
•464•1•0•1•Updated Jun 18, 2025Jun 18, 2025
vllm-flash-attention
Public
Fast and memory-efficient exact attention
Python
•
BSD 3-Clause "New" or "Revised" License
•1.9k•5•0•0•Updated Jun 11, 2025Jun 11, 2025
pytest-nm-releng
Public
Pytest plugin used by the Release Engineering team
Python
•
Apache License 2.0
•0•0•0•0•Updated Jun 9, 2025Jun 9, 2025
lm-evaluation-harness
Public
A framework for few-shot evaluation of language models.
Python
•
MIT License
•2.6k•4•0•1•Updated Jun 9, 2025Jun 9, 2025
yolov5
Public archive
YOLOv5 in PyTorch > ONNX > CoreML > TFLite
Python
•
GNU General Public License v3.0
•17k•19•0•0•Updated Jun 4, 2025Jun 4, 2025
yolov3
Public archive
YOLOv3 in PyTorch > ONNX > CoreML > TFLite
Python
•
GNU General Public License v3.0
•3.5k•3•0•0•Updated Jun 4, 2025Jun 4, 2025
transformers
Public archive
🤗Transformers: State-of-the-art Natural Language Processing for Pytorch and TensorFlow 2.0.
Python
•
Apache License 2.0
•30k•9•0•0•Updated Jun 4, 2025Jun 4, 2025
llm-d
Public
llm-d is a Kubernetes-native high-performance distributed LLM inference framework
Makefile
•
Apache License 2.0
•130•0•0•0•Updated Jun 3, 2025Jun 3, 2025
deepsparse
Public archive
Sparsity-aware deep learning inference runtime for CPUs
nlp performance computer-vision inference machinelearning pruning object-detection pretrained-models quantization cpus
Python
•
Other
•190•3.2k•1•0•Updated Jun 2, 2025Jun 2, 2025
sparsify
Public archive
ML model optimization product to accelerate inference.
computer-vision tensorflow keras pytorch image-classification pruning object-detection quantization automl onnx
Python
•
Apache License 2.0
•30•326•1•0•Updated Jun 2, 2025Jun 2, 2025
sparseml
Public archive
Libraries for applying sparsification recipes to neural networks with a few lines of code, enabling faster and smaller models
sparsity keras deep-learning-algorithms deep-learning-library pruning object-detection computer-vision-algorithms onnx deep-learning-models sparsification
Python
•
Apache License 2.0
•157•2.1k•1•0•Updated Jun 2, 2025Jun 2, 2025
docs
Public archive
Top-level directory for documentation and general content
MDX
•7•121•0•0•Updated Jun 2, 2025Jun 2, 2025
sparsezoo
Public archive
Neural network model repository for highly sparse and sparse-quantized models with matching sparsification recipes
nlp computer-vision deep-learning-algorithms yolo resnet pruning transfer-learning pretrained-models quantization mobilenet
Python
•
Apache License 2.0
•28•392•1•0•Updated Jun 2, 2025Jun 2, 2025
gateway-api-inference-extension
Public archive
Gateway API Inference Extension
Jupyter Notebook
•
Apache License 2.0
•136•1•0•0•Updated May 14, 2025May 14, 2025
lighteval
Public
Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends
Python
•
MIT License
•320•0•0•0•Updated May 7, 2025May 7, 2025
AutoFP8
Public
Python
•
Apache License 2.0
•26•195•12•3•Updated May 5, 2025May 5, 2025