MLCommons

All

102 repositories

mlperf_inference_test_submissions_v5.0
Public
HTML
•2•0•0•0•Updated Jan 8, 2025Jan 8, 2025
modelbench
Public
Run safety benchmarks against AI models and view detailed reports showing how well they performed.
Python
•
Apache License 2.0
•11•71•177•3•Updated Jan 8, 2025Jan 8, 2025
inference
Public
Reference implementations of MLPerf™ inference benchmarks
benchmark machine-learning
Python
•
Apache License 2.0
•539•1.3k•167•15•Updated Jan 7, 2025Jan 7, 2025
logging
Public
MLPerf™ logging library
Python
•
Apache License 2.0
•46•32•41•2•Updated Jan 7, 2025Jan 7, 2025
inference_policies
Public
Issues related to MLPerf™ Inference policies, including rules and suggested changes
Apache License 2.0
•52•58•18•2•Updated Jan 7, 2025Jan 7, 2025
mobile_app_open
Public
Mobile App Open
C++
•
Apache License 2.0
•24•50•29•2•Updated Jan 7, 2025Jan 7, 2025
mobile_open
Public
MLPerf Mobile benchmarks
Python
•
Apache License 2.0
•2•10•6•2•Updated Jan 7, 2025Jan 7, 2025
mlperf-automations
Public
Python
•
Apache License 2.0
•4•1•47•0•Updated Jan 6, 2025Jan 6, 2025
training
Public
Reference implementations of MLPerf™ training benchmarks
benchmark machine-learning
Python
•
Apache License 2.0
•562•1.6k•18•43•Updated Jan 6, 2025Jan 6, 2025
inference_results_visualization_template
Public template
JavaScript
•
Apache License 2.0
•6•0•1•1•Updated Jan 4, 2025Jan 4, 2025
mlperf_inference_unofficial_submissions_v5.0
Public
These are automated test submissions for validating the MLPerf inference workflows
mlperf mlperf-inference mlperf-inference-results
Mermaid
•1•0•0•0•Updated Jan 4, 2025Jan 4, 2025
ailuminate
Public
Apache License 2.0
•1•0•0•0•Updated Jan 4, 2025Jan 4, 2025
GaNDLF-Synth
Public
Extension for GaNDLF [gandlf.org] to enable synthesis.
medical-imaging synthesis
Python
•
Apache License 2.0
•2•4•6•0•Updated Dec 30, 2024Dec 30, 2024
tiny
Public
MLPerf™ Tiny is an ML benchmark suite for extremely low-power systems such as microcontrollers
C
•
Apache License 2.0
•92•366•13•6•Updated Dec 24, 2024Dec 24, 2024
medperf
Public
An open benchmarking platform for medical artificial intelligence using Federated Evaluation.
Python
•
Apache License 2.0
•33•150•52•23•Updated Dec 21, 2024Dec 21, 2024
algorithmic-efficiency
Public
MLCommons Algorithmic Efficiency is a benchmark and competition measuring neural network training speedups due to algorithmic improvements in both training algorithms and models.
Python
•
Apache License 2.0
•69•344•30•11•Updated Dec 21, 2024Dec 21, 2024
dynabench
Public
Python
•
MIT License
•16•21•7•4•Updated Dec 20, 2024Dec 20, 2024
mlperf_inference_submissions
Public template
Shell
•0•0•0•0•Updated Dec 20, 2024Dec 20, 2024
ck
Public
Collective Knowledge (CK) and Collective Minds (CM): community-driven projects to learn how to run AI, ML and other emerging workloads in a more efficient and cost-effective way across diverse models, datasets, software and hardware using MLPerf, CM and CK
education automation optimization best-practices collaboration cm modularity workflows portability reusability
Python
•
Apache License 2.0
•120•613•14•0•Updated Dec 20, 2024Dec 20, 2024
GaNDLF
Public
A generalizable application framework for segmentation, regression, and classification using PyTorch
machine-learning framework deep-learning regression pytorch medical-imaging classification segmentation data-augmentation biomedical-image-processing
Python
•
Apache License 2.0
•80•169•15•2•Updated Dec 20, 2024Dec 20, 2024
policies
Public
General policies for MLPerf™ including submission rules, coding standards, etc.
Python
•
Apache License 2.0
•55•28•35•39•Updated Dec 18, 2024Dec 18, 2024
power-dev
Public
Dev repo for power measurement for the MLPerf™ benchmarks
Python
•
Apache License 2.0
•25•19•10•4•Updated Dec 17, 2024Dec 17, 2024
training_results_v4.1
Public
This repository contains the results and code for the MLPerf™ Training v4.1 benchmark.
Python
•6•4•1•0•Updated Dec 11, 2024Dec 11, 2024
mlperf-automations_archived
Public archive
This repository contains the automations and scripts used to automate MLPerf benchmarks (mainly MLPerf inference for now)
automation mlperf
Python
•
Apache License 2.0
•25•0•46•0•Updated Dec 11, 2024Dec 11, 2024
mlperf_client
Public
MLPerf Client is a benchmark for Windows and macOS, focusing on client form factors in ML inference scenarios.
benchmarking machine-learning ml
C++
•
Apache License 2.0
•0•8•0•0•Updated Dec 10, 2024Dec 10, 2024
ck-mlops
Public
A collection of portable workflows, automation recipes and components for MLOps in a unified CK format. Note that this repository is outdated - please check the 2nd generation of the CK workflow automation meta-framework with portable MLOps and DevOps components here:
machine-learning automation optimization artificial-intelligence open-science systems reproducibility workflows portability co-design
Python
•
Apache License 2.0
•7•18•0•0•Updated Dec 10, 2024Dec 10, 2024
cm4mlops
Public
A collection of portable, reusable and cross-platform CM automations to modularize MLPerf and simplify the process of building and benchmarking AI systems across diverse models, data sets, software and hardware
workflow devops machine-learning automation cross-platform reusable workflow-automation mlops mlperf
Python
•
Apache License 2.0
•25•16•7•0•Updated Dec 10, 2024Dec 10, 2024
chakra
Public
Repository for MLCommons Chakra schema and tools
Python
•
Apache License 2.0
•40•77•27•11•Updated Dec 6, 2024Dec 6, 2024
croissant
Public
Croissant is a high-level format for machine learning datasets that brings together four rich layers.
machine-learning schema-org json-ld datasets
Python
•
Apache License 2.0
•45•484•124•19•Updated Dec 5, 2024Dec 5, 2024
datasets-knowledge-graphs
Public
Apache License 2.0
•0•0•0•0•Updated Dec 3, 2024Dec 3, 2024