MLPerf Training Introduces Its First LLM Post-Training Benchmark
MLCommons ·
MLPerf Training v6.1 adds an agentic reinforcement-learning workload that measures how quickly systems can teach a 397-billion-parameter language model to repair real software projects. The post MLPerf Training Introduces Its First LLM Post-Training Benchmark appeared first on MLCommons .
MLPerf Training is adding a new LLM Post-Training benchmark beginning with the v6.1 submission round in October 2026, complementing the existing suite of pre-training benchmarks. While pre-training builds foundational intelligence via massive-scale data ingestion, post-training is a complementary process that refines that foundation to excel at specific tasks. Since the second half of 2025, significant advances in LLM training have come from scaling post-training. Two great examples of this are small models starting to approach frontier-level performance ( Qwen 3.8 27B ) or generational improvements while using the same base model ( “Scaling post-training is all we did for GLM-5.3” ).
The MLPerf Reasoning Task Force came together to deliver an MLPerf Training benchmark that can complete a real-world LLM post-training workload in under 1024 GB300 hours. Real-world post-training combines techniques such as distillation, reinforcement learning, and supervised fine-tuning across domains including math, reasoning, and agentic tasks. After surveying the post-training landscape, we selected the Reinforcement Learning with Verifiable Rewards (RLVR) method because it is well suited to software tasks. An LLM-powered coding agent solves agentic software engineering (SWE) tasks in a sandbox environment; each attempt to complete a task is called a rollout. For each problem, the system samples multiple rollouts and assigns a binary pass/fail reward. The resulting traces are then used in Group Relative Policy Optimization (GRPO), which compares rewards within each group and updates the model to make higher-reward solutions more likely. The updated model weights are then used in the next iteration of rollouts.
Developing this benchmark required the most complex system-level integration in MLPerf Training to date. It requires deploying three different components: training, inference, and agent environments. At runtime, hundreds of concurrent rollouts contend for CPU time, GPU time, memory, and storage, and resources must be balanced between them. State-of-the-art training techniques are also particularly sensitive to numerics.
The model selected for this benchmark is Qwen 3.5 397B, the largest model in the Qwen 3.5 family released in February 2026. It is available as an open-weight model under the Apache 2.0 license. Qwen 3.5 is a mixture-of-experts (MoE) model with 397 billion total parameters and 17 billion active parameters per token. Qwen 3.5 was the first model family to productize a hybrid architecture consisting of Gated DeltaNet (GDN) and sparse MoE layers. Each MoE layer activates 1 shared expert and only 10 of 512 routed experts (2%). GDN is a time- and space-efficient alternative to standard attention that shows strong in-context learning abilities. In particular, GDN replaces the linear-growth KV cache with a fixed-size, compressed state. It conceptually learns past associations rather than explicitly storing them, and the fixed size is particularly valuable for long context. Both MoE and GDN reflect current trends in model architectures.
The MLPerf Training benchmarks measure time-to-train to a pre-determined accuracy metric as the score. This metric ensures the benchmark isn’t just a raw throughput measure; it measures an objective that mimics real-world training. More casually, the benchmarks require an answer that is both correct and fast . Benchmarks that measure only throughput can be misleading by encouraging performance optimizations that hurt final model quality and therefore don’t help real-world training. For example, reduced-precision numerical formats can improve throughput but may not provide enough numerical stability to achieve target accuracy. A more subtle example is increasing policy lag (in the asynchronous scenario), which may increase throughput for one system but degrade model quality. For this reason, AI benchmarks need to account for trained model quality in addition to throughput. The MLPerf benchmarks enable submitters to tune their systems for optimal performance while also keeping model quality in check.
Applying this philosophy to post-training is challenging. ll large parameter count open-weight models are already post-trained on most publicly available post-training datasets. As a result, developing a benchmark that can improve accuracy on a public dataset in a limited training time is quite difficult. The published weights for Qwen3.5-397B-A17B feature post-training, which makes the model capable of most agentic tasks. So this benchmark aims to slightly improve the model’s performance specifically on SWE tasks.
We thank the Qwen Team for providing this model to the community. Furthermore, we welcome and encourage other members of the community to publish open-weight models at different stages of training to support public benchmarking efforts.
The benchmark uses the Apache 2.0-licensed R2E-Gym dataset of SWE problems. This is one of the larger datasets of procedurally generated executable environments, created from GitHub commits of software projects in Python: Aiohttp, Datalad, NumPy, Pandas, Pyramid, Pillow, Sympy, and Tornado. The agent is given a Linux container environment with a Git repository, all dependencies pre-installed, and an issue description in the prompt. The task is to come up with a patch that would solve the problem.
While searching for a stable operating regime for the benchmark, we imposed restrictions on max number of agent turns and available max context length. To fit those constraints, we selected a subset of the R2E-Gym dataset based on proxies for difficulty such as the number of modified lines and files in the “golden” solution and the ability of the post-trained model to solve the tasks. The result is a dataset consisting of 700 training problems and a non-overlapping set of 251 validation problems.
For post-training, the dataset includes prompts and container images, published on MLCommons storage in Apptainer SIF format. We faced interesting challenges when preparing the dataset: the original R2E-Gym container images were built for x86_64 in early 2025. Rebuilding them in 2026 for Aarch64 resulted in images that worked slightly differently, mostly due to dependency drift, updated packages, and architecture-dependent packages. Consequently, some of the problems could not be solved. We audited the dataset and fixed any such issues when possible or removed problems from the dataset to ensure that all of our environments work the same way across architectures and are solvable.
We would like to thank the R2E-Gym authors for providing this dataset to the community.
The model interacts with the sandbox environment through a harness. For a given context length and turn limit, the choice of harness determines how efficiently the model uses those budgets. We found that the choice of harness strongly influenced solve rate, which translates directly to training performance because RLVR uses sparse binary rewards. We selected the OpenHands harness for both training and validation based on its performance on the dataset. During RL training, models sometimes find ways to obtain a reward without solving the underlying task, for example by removing failing tests rather than fixing the bugs causing the test failure. This is called “reward hacking” and teaches the model solutions that score well in training but fail to generalize. To prevent reward hacking, the agent evaluation uses test files the agent can’t see or modify during execution.
To simulate extended multi-turn agentic work, the task force set a maximum context length of 65536, a maximum of 30 agent turns, and 16 generations per prompt. We chose these constraints to balance training performance with duration. A larger context requires more resources and enables stronger model performance, making it harder to achieve quality improvements. Conversely, a smaller context can hinder the model in solving tasks, which affects rewards and prevents GRPO from finding a training signal. The training objective is best characterized as optimizing the model’s ability to operate efficiently under the limited context/turn budget.
The model is configured in instruction-following mode with reasoning mode disabled, but the model has access to a “think” tool through the OpenHands harness. Multi-turn agentic rollouts with long context result in a workload that heavily exercises the KV cache management system of the generation endpoints. A balance must be struck between keeping the KV cache in GPU memory between turns and re-running prefill for previously processed context. Submitters will need to find that balance.
The reference implementation of the benchmark is built on NVIDIA NeMo-RL and separates the pool of worker nodes into policy training and rollout generation workers. Agent sandboxes are executed on the same GPU nodes. Ray coordinates the distributed components:
The reference utilizes asynchronous training, which is important in making RL performant. Under async RL, training and generation can overlap, resulting in stale (off-policy) model weights for generation. The maximum staleness, or age, of trajectories used for training is 1 weight version; submissions are free to use different values.
The disaggregated design of the reference highlights a central challenge in asynchronous reinforcement learning: to achieve optimal performance, training and inference must be rate-matched and optimized together. Optimizing only one part of this pipeline simply moves the bottleneck elsewhere.
When evaluating model performance, we found high variability in the solve rate even with deterministic sampling (temperature=0.0). The model could solve a task on one attempt and fail on the next. Multi-turn agentic trajectories are never deterministically reproducible due to tool calling, timing, and batching effects. One approach to achieving a more stable evaluation score is pass@k.
Under pass@k, each problem is evaluated k times, and we measure how many problems are solved at least once out of k times. For this benchmark, we settled on the pass@4 metric as the validation metric, which makes validation consist of 1004 rollouts. The benchmark objective identified in the course of the reference convergence point study is to reach a pass@4 score of 0.69 – to solve 69% of validation problems at least once in 4 tries. Token sampling for validation uses temperature=0.1 and top_p=0.95. The chart below shows the dispersion of the metrics we evaluated: pass@4, pass@1-average-of-4, and pass^4 (all 4 attempts must pass). Pass@4 shows the lowest coefficient of variance.
Agentic reinforcement learning is significantly less deterministic than pre-training. The training data is not static, and a built-in feedback loop updates weights used for future rollouts, which in turn affect future training steps. At the trajectory level, small numerical differences change sampled tokens; those differences alter tool calls and intermediate outputs. These effects compound over many turns. Other sources of non-determinism include request routing, different training/generation backends, numerical sensitivity in MoE routing, non-determinism in executed kernels, and off-policy training. This variability is expected in the learning process built around exploration, but it complicates a benchmark that needs repeatable quality targets.
Evaluation is also expensive. Unlike a forward-only validation pass, each evaluation launches complete software-engineering agents that generate tokens, use tools, and execute tests. In the reference setup, one validation requires 1004 trajectories and can cost several training steps.
The benchmark addresses these constraints in three ways. First, grouped pass@4 makes the quality metric more stable. We also fixed the traversal order of the training dataset according to a difficulty-aware ordering to reduce variability. Finally, the benchmark rules delay the first full evaluation until the model is expected to be near the target. The reference qualified global batch sizes of 256, 512, and 1024 begin validation at 4608, 5120, and 7168 training samples, respectively, then evaluate after each later step. These evaluation starting points follow an empirical scaling law determined during the study of the reference’s convergence behavior: GBS * ceil(2.5 + 3840 / GBS). This schedule is meant to ensure that all runs converge to their target at the first evaluation.
This MLPerf Training benchmark reports the wall-clock time required to reach the fixed pass@4 quality target. The reference convergence point study includes global batch sizes of 256, 512, and 1024, with batch-dependent learning rates, gradient norm clipping, and first-validation steps. The study was performed on GB300 NVL72 racks, using 64 compute nodes with 4 Blackwell Ultra GPUs each (256 GPUs total) and BF16 for both training and generation, and required between 140 and 160 minutes per run, a baseline submitters are expected to improve on. These settings define the qualified reference points; submitters may select other global batch sizes and remain responsible for following the MLPerf Training rules that govern legal configurations, optimizations, and alternative implementations.
The LLM post-training benchmark brings several previously separate systems concerns into one measured workload: very large mixture-of-experts policy training, high-throughput inference, frequent policy-weight transfers, asynchronous trajectory collection, and long multi-turn trajectories with heavy KV cache load.
That combination creates new optimization opportunities and challenges. Submitters can improve computation, communication, scheduling, rollout throughput, environment execution, and the balance between training and generation resources, but must maintain numerical stability and avoid training collapse. Because the benchmark measures the complete time-to-quality path, improvements must accelerate the system without sacrificing the agent’s ability to solve unseen tasks.
MLPerf Training’s LLM post-training benchmark reflects a major shift in how leading large language models gain useful capabilities. The workload does not stop at predicting the next token or adapting on a static dataset, but requires the model to learn how to take the right actions on real software-engineering tasks.
By establishing a common model, task, dataset, quality target, and evaluation procedure, MLCommons gives the industry a practical way to measure progress in agentic reinforcement-learning systems. The benchmark will help expose where time and resources are spent—and encourage advances across the full post-training pipeline.
The benchmark specification and reference implementation are available on GitHub .
We would like to thank the entire MLPerf Training Reasoning Task Force for coming together and shaping this benchmark.
MLCommons is the world’s leader in AI benchmarking. An open engineering consortium supported by over 125 members and affiliates, MLCommons has a proven record of bringing together academia, industry, and civil society to measure and improve AI. MLCommons began with the MLPerf benchmarks in 2018, which quickly grew into a set of industry metrics for measuring machine learning performance and promoting transparency in machine learning techniques. Since then, MLCommons has continued to use collective engineering to build the benchmarks and metrics required for better AI – ultimately helping to evaluate and improve the accuracy, safety, speed, and efficiency of AI technologies.
For additional information on MLCommons and details on becoming a member, please visit MLCommons.org or email participation@mlcommons.org.
The post MLPerf Training Introduces Its First LLM Post-Training Benchmark appeared first on MLCommons .