
Nonprofit · San Francisco, United States
MLCommons
A vendor-neutral consortium that builds and publishes the MLPerf benchmarks, open datasets and AI safety measurements used across the industry.
San Francisco, United States
- Type
- Nonprofit
- Based in
- San Francisco, United States
- Key papers
- 2
- Featured projects
- 7
About
MLCommons grew out of MLPerf, a benchmarking effort begun in 2018 after engineers and researchers from Baidu, Google, Harvard, Stanford and UC Berkeley began meeting to agree on common ways to measure machine learning performance. The MLCommons association was launched in December 2020 to give that work, and related dataset and tooling projects, a permanent home.
Its working groups publish benchmark suites for training, inference, high-performance computing, mobile and tiny devices, together with reference implementations on GitHub. Datasets such as People's Speech and tooling such as MLCube are released openly.
Since 2023 an AI Risk and Reliability group has built safety measurement, which produced the AILuminate benchmark for large language models in December 2024. Member companies, universities and non-profits fund and govern the consortium, and its results are widely cited by hardware vendors and cloud providers.
Mission and ambitionsMLCommons states its mission as standardizing AI measurement so that buyers, builders and policymakers can make informed decisions about accuracy, safety, speed and efficiency.
Known for
- 01
MLPerf training and inference benchmarks
- 02
People's Speech open speech dataset
- 03
AILuminate LLM safety benchmark
- 04
MLCube portable ML packaging
Milestones
- 2018
MLPerf Training launched
Working group announced the first MLPerf Training benchmark in May after early meetings among industry and academic engineers.
- 2019
MLPerf Inference launched
The inference benchmark was announced in June, with first results published in November.
- 2019
MLCube created
A packaging standard for portable machine learning workloads was introduced in November.
- 2020
MLCommons association launched
The consortium was formally launched on 3 December, absorbing MLPerf and the People's Speech effort.
- 2023
AI Risk and Reliability group formed
A working group on AI safety measurement began in November.
- 2024
AILuminate v1.0 released
The first benchmark for assessing the safety of general-purpose chat models was released on 4 December in English.
Open source
MLPerf training.
MLPerf inference.
Croissant.
Collective Knowledge.
- peoples_speechdataset
People's Speech.
- unsupervised_peoples_speechdataset
Unsup.
- ml_spoken_wordsdataset
Spoken Words.
Research papers
Notable releases
Openness
Open code and datasets; licences vary.
People
- Peter MattsonPresident and founder
- Rebecca WeissExecutive Director
- David KanterHead of MLPerf
- Vijay Janapa ReddiVice President
- Carole-Jean WuVice President
Listed from the lab's own pages and public profiles.
Contact and community
Contact details and links are on the lab's website (https://mlcommons.org); code questions go through issues on their GitHub organisation.
Sources
Compiled from the lab's public pages, Hugging Face and GitHub. Last checked 2026-11-17. Something wrong or out of date? Tell us on Discord.



