# Olmo-core 3 opens Ai2's MoE training stack, with bounded GPU results

> Ai2's October 1 Olmo-core 3 release provides open MoE training code and B300 benchmarks. Its scale tests measure systems throughput, not a trained model's quality.

By BIG CHANGE Editorial

Published: 2026-10-13T02:08:18.598Z
Updated: 2026-10-13T02:08:18.598Z
Canonical: https://bigchange.ai/blog/ai2-olmo-core-3-moe-training-stack-benchmarks

![Conceptual charcoal illustration of an unbranded accelerator circuit board beside its separate finned heat sink.](https://bigchange.ai/api/media/file/olmocore3-gpu-board-hero-v2.png)
AI-generated conceptual illustration by BIG CHANGE. Generic accelerator components; no specific B300 design, Ai2 installation, completed training run or BIG CHANGE hardware test is depicted.

Ai2 released Olmo-core 3 on October 1 as an open training framework for mixture-of-experts (MoE) language models. Its most useful result for a systems engineer is narrower than the trillion-parameter headline: Ai2 has published the code, reference configurations and bounded GPU measurements needed to decide whether this stack merits a trial on a particular cluster.

An MoE sends each token to a subset of its expert networks. That limits computation per token, but the full pool of experts and their optimizer state still need space across GPUs. Moving tokens to those experts also costs time. Olmo-core 3 changes how Ai2 handles those two constraints: its DDP-based system leaves experts resident on GPUs and routes data to them, where the earlier FSDP-based implementation repeatedly gathered and resharded weights for microbatches. The new stack combines expert and pipeline parallelism, distributed optimizer state, GPU-resident routing and grouped expert computation. Ai2 also supports MXFP8 for selected operations. These are design choices a team can inspect and configure, not a promise of the same speed on another cluster.

## The big change

Olmo-core 3 makes Ai2's large MoE training machinery available as Apache 2.0 code and reference scripts. Ai2's tests show higher throughput than its earlier stack in a specific eight-B300 comparison and show that a random-routed 1.2-trillion-parameter configuration can execute across 512 B300s. They do not establish the quality of a trained trillion-parameter model or the speed of a reader's workload. The practical gain is an inspectable starting point for reproducing the configurations, measuring a local bottleneck and deciding which parallelism and precision settings are worth a costly training run.

## Three figures, three denominators

Ai2's preliminary 47-billion-parameter MoE comparison ran on eight Nvidia B300 GPUs. The new stack processed 52,000 tokens per second **per GPU**, versus 19,400 with its earlier FSDP-based implementation. The ratio is 52,000 ÷ 19,400 = 2.68, rounded to 2.7 times. This is an internal old-versus-new *stack* comparison, not a measurement against Megatron-Core or a clean DDP-only ablation. The tagged examples' closest DDP+expert-parallel versus FSDP/HSDP comparison holds model shape and measurement window steady, but changes the router, sparse kernels, optimizer and other implementation details too.

The “less than 5%” result asks a different question. Ai2 increased the pool from 8 to 128 experts while selecting four per token. Total capacity rose from 4.6 billion to 47 billion parameters, while active parameters per token stayed around 3.2 billion. Throughput fell by less than 5% **between those expert-pool configurations**. It does not say the new stack is only 5% slower than the old one, or that adding 16 times as many experts is free in every setup. The release does not give enough information to turn that percentage into a reader's dollar cost.

The scale result has a third denominator: Ai2 reports a highest observed 858 TFLOP/s **per GPU** for a 1.2-trillion-total-parameter configuration, with 58.36 billion parameters active per token, on 512 B300 GPUs. The v3.0.0 reference table identifies this Ultra-128E row as random-routed, MXFP8 and per-layer recomputation. Random routing measures the system path without testing whether a learned router produces a good model. A 2.38-trillion-parameter configuration used DeepEP v2 in a short capacity test. Ai2 describes it as a scale check, not sustained training. Neither number is a trained model's accuracy, useful output quality, or evidence of stable full-run economics.

Ai2 also reports a controlled four-B300 comparison in which MXFP8 raised end-to-end training throughput about 21% against BF16 and lowered peak active memory from 103 GiB to 95 GiB, with work spread uniformly across experts. That result concerns the tested precision choices and workload. It cannot be added to the 2.7-times figure as if the measurements shared one baseline.

## What an engineer can actually try

The repository's October 1 `v3.0.0` release is public under Apache 2.0. The README says to install PyTorch for the target operating system and hardware first, then install `ai2-olmo-core` from PyPI or install the cloned repository for development. The tagged `src/examples/olmo_ddp/README.md` maps reported operating points to scripts and documents `torchrun` as the generic launch path. Its production scripts assume eight GPUs per node and specify their node count in `REF_NUM_NODES`. These are examples for people who already have a distributed GPU environment, not a one-command route to reproducing Ai2's B300 results.

The report scripts have material access requirements. Their README names the `beaker` extra, FlashAttention 4, a writable `/workspace`, and read access to Ai2's `s3://ai2-llm` data mix; the DeepEP v2 path additionally needs a DeepEP checkout. Ai2's main README says its published Docker images contain dependencies but not the Olmo-core package, and warns that different hardware or driver/CUDA versions may need an adapted image. The v3.0.0 release raises the minimum supported PyTorch version to 2.10.0. We reviewed these documents and the tagged reference table; BIG CHANGE did not install the package, gain access to the data mix, reserve GPUs or run a training job.

For a local evaluation, start with the tagged reference row closest to the intended model size and available hardware, then account for every dependency and inaccessible data path before reserving compute. Compare per-GPU tokens per second under the same sequence length, batch, routing, precision and measurement window; separately monitor peak memory and checkpoint behavior. The published production rows use sequence length 8,192, top-four routing and one shared expert. Tiny and Small have learned-routing companion scripts, whereas the headline larger rows are random-routed. A team's learned-routing data distribution can change communication balance and throughput, so a representative pilot remains necessary before projecting a full run.

The cost boundary is physical as well as software. The featured eight-GPU comparison requires eight B300s; the largest throughput result used 512. Ai2 provides no general hourly rental price or full training budget in the release. An Apache 2.0 license removes a software licensing fee for this code, but it does not provide GPUs, compatible drivers, the named data access or operator time. The reference README also says the report's two-batch-overlap measurement used a pre-merge branch and cannot be reproduced from the tagged scripts.

The open system is useful because its claims can be checked against actual configurations. Ai2's release summarizes cases where an apparently better routing-balance score hid less balanced work, and where overlapping communication with computation slowed training. Those findings make the local measurement plan more important: a fast kernel or favorable single metric is not the end-to-end training result. Ai2 says its next Olmo generation will use an MoE architecture; this release offers the training stack and system measurements ahead of any quality assessment of that future model.

## Sources & further reading

- [Ai2's October 1 release](https://allenai.org/blog/olmocore3) gives the architecture explanation and the bounded throughput, memory and scale figures. The figures are Ai2 measurements; BIG CHANGE did not reproduce them.
- [Olmo-core v3.0.0 release](https://github.com/allenai/Olmo-core/releases/tag/v3.0.0) establishes the tagged version, October 1 release, PyTorch minimum and code changes. [The repository README](https://github.com/allenai/Olmo-core) documents installation, optional dependencies and Docker image caveats.
- [Tagged MoE reference-script guide](https://raw.githubusercontent.com/allenai/Olmo-core/v3.0.0/src/examples/olmo_ddp/README.md) maps report configurations to GPU count, routing and precision, and explains the limits of the old-versus-new stack comparison. [The technical report](https://allenai.org/papers/olmocore3) is the primary account of the systems experiments; its PDF could not be retrieved in our research environment, so detailed claims here rely on Ai2's accessible release and tagged scripts.

## Sources

- [Ai2, Introducing Olmo-core 3](https://allenai.org/blog/olmocore3) — Primary architecture and measured-result announcement; Ai2-reported figures.
- [Olmo-core v3.0.0 release](https://github.com/allenai/Olmo-core/releases/tag/v3.0.0) — Tagged release, version, October 1 release date, PyTorch minimum and code change record.
- [Olmo-core v3.0.0 MoE reference-script README](https://raw.githubusercontent.com/allenai/Olmo-core/v3.0.0/src/examples/olmo_ddp/README.md) — Operating points, random routing, GPU counts, prerequisites and comparison limits.
- [Olmo-core repository README and license](https://github.com/allenai/Olmo-core) — Installation, optional dependencies, Docker caveats and Apache 2.0 license.
- [Ai2 technical report](https://allenai.org/papers/olmocore3) — Linked primary report; full PDF retrieval failed; no unique technical detail attributed to unread PDF.