Standards consortium · Funders · Dover, Delaware · Est. 2020
MLCommons is a non-profit, open AI engineering consortium focused on collaborative, open benchmark and dataset work intended to improve AI systems’ quality, performance, safety, speed, and efficiency. The organization coordinates multi-stakeholder efforts across industry and academia to publish peer-reviewed, industry-standard benchmark suites and supporting tooling—most notably the MLPerf family—and to build dataset and engineering standards that help make ML evaluation and data development more reproducible and comparable.
Its governance emphasizes technical, hands-on leadership and consensus-supported decisions, with a board drawn from the MLCommons community. MLCommons also runs working groups that extend benchmarking to new areas such as AI risk & reliability and medical AI, and develops shared data standards (for example, Croissant) intended to make datasets easier to discover, govern, and use across tools and platforms.
No curated programs yet. Check the careers page ↗.
A $3.5 million, two-year award will fund an industry-standard benchmark for the efficiency of complete agentic AI workflows The post MLCommons Selected as Benchmarking Team for U.S. National Science Foundation–Backed AI Efficiency Challenge to Build MLPerf Agents appeared first o
What is MLPerf Endpoints, and why it matters to buyersA buyer's guide to MLPerf Endpoints: the full operating frontier, five metrics that map to decisions, and how to read a result. The post What is MLPerf Endpoints, and why it matters to buyers appeared first on MLCommons .
Our mission, updated: standardize AI measurement and enable informed decisionsPeter Mattson, President, on MLCommons' updated mission - and the changes ahead The post Our mission, updated: standardize AI measurement and enable informed decisions appeared first on MLCommons .
Open, peer-reviewed measurement for today’s AI buyers: MLPerf in the EU’s AI Gigafactories tender, and Endpoints v1.0 opens for submissionsRolling submissions, mandatory power reporting, and a place in the EU’s AI Gigafactories tender: measurement buyers can trust. The post Open, peer-reviewed measurement for today’s AI buyers: MLPerf in the EU’s AI Gigafactories tender, and Endpoints v1.0 opens for submissions appe
Assessing & Managing AI Risk for Business and Commercial Deployments: A business owner’s guide to a cost you can measure, price, and manageTeams managing AI risk are stuck between protecting the business and adopting it faster. Independent measurement reconciles the two. The post Assessing & Managing AI Risk for Business and Commercial Deployments: A business owner’s guide to a cost you can measure, price, and manag
What Could Go Wrong When Agents Handle Your Sensitive Data? A New Taxonomy Answers.Agents create privacy risks chatbots do not. The MLCommons Privacy Working Group's new taxonomy maps five areas of agentic data protection risk - and v0.1 is open for feedback. The post What Could Go Wrong When Agents Handle Your Sensitive Data? A New Taxonomy Answers. appeared f
DLRMv4: The Next-Generation Recommendation MLPerf Training BenchmarkEvolving MLPerf Training from feature-interaction baselines to production-scale sequential recommendation with HSTU and the 560 GB Yambda-5B embedding benchmark. The post DLRMv4: The Next-Generation Recommendation MLPerf Training Benchmark appeared first on MLCommons .
MLPerf Training Introduces Its First LLM Post-Training BenchmarkMLPerf Training v6.1 adds an agentic reinforcement-learning workload that measures how quickly systems can teach a 397-billion-parameter language model to repair real software projects. The post MLPerf Training Introduces Its First LLM Post-Training Benchmark appeared first on ML
MLCommons Joins EU-Funded AIRIS Project to Build and Benchmark Next-Generation Biomedical AIStandardizing independent evaluation, fairness, and interpretability for mechanism-informed generative AI in healthcare The post MLCommons Joins EU-Funded AIRIS Project to Build and Benchmark Next-Generation Biomedical AI appeared first on MLCommons .
Where the Industry Is Investing: A Look at MLPerf Inference v6.1In this analysis, MLPerf Inference Working Group chairs Miro Hodak and Frank Han break down a record field of 30 submitters and 120 systems, the arrival of agentic and end-to-end benchmarks, and what the results signal about where inference engineering is headed. The post Where t
MLCommons Sets Participation Record with New MLPerf Inference v6.1 Benchmark ResultsLatest version of the benchmark introduces two new tests for emerging AI deployment patterns, including Agentic Inference The post MLCommons Sets Participation Record with New MLPerf Inference v6.1 Benchmark Results appeared first on MLCommons .
MLCommons Agent Reliability Profile Named a Finalist in Global Agentic Regulator HackathonThe Financial Services Working Group's flagship initiative advances to the final round of the C:>DIR Global "Agentic Regulator" Hackathon, with winners announced September 18. The post MLCommons Agent Reliability Profile Named a Finalist in Global Agentic Regulator Hackathon appe
MLCommons announced new MLPerf Inference v6.1 results, stating it set a participation record and adding tests aligned with emerging AI inference deployment trends.
MLCommons Releases New MLPerf Storage v3.0 Benchmark ResultsMLCommons announced MLPerf Storage v3.0 results, describing new test coverage for modern storage workloads and adding S3 object storage access layer support plus new KV Cache and Vector Database test types.
MLPerf Endpoints v0.7: A Foundation ReleaseMLCommons announced MLPerf Endpoints v0.7, including new results, automated submission pipeline capabilities, and a roadmap toward further iteration.
MedPerf + Google Cloud: Secure Medical AI BenchmarkingMLCommons and Google Cloud announced enabling MedPerf on Google Cloud confidential compute capabilities for medical AI benchmarking using federated evaluation concepts.
MLCommons Chakra: Standardizing AI Systems Co-Design (Chakra comes of age)MLCommons reported on Chakra’s maturation, describing how it provides a shared framework for understanding and reproducing workload behavior across hardware/software/infrastructure boundaries for AI systems co-design.
MLCommons Releases New MLPerf Inference v6.0 Benchmark ResultsMLCommons released MLPerf Inference v6.0 results, describing new or updated datacenter/edge tests and new LLM and reasoning benchmark components.