Microsoft has spent years buying Nvidia accelerators by the shipping container, which makes a chip of its own design something of an occasion. The company has introduced Maia 200, an accelerator built for inference, and the first racks are already running in a data center region near Des Moines, Iowa.
Inference is the unglamorous half of artificial intelligence. Training is the apprenticeship, months of expensive study; inference is the day job, answering one question after another for millions of people who expect a reply before they lose interest. The economics of the entire business live here, because every answer burns electricity, and Maia 200 exists to make each answer cost less.
Start with the silicon. Fabricated on TSMC's 3-nanometer process, each chip carries more than 140 billion transistors inside a 750-watt power envelope. Its headline arithmetic is more than 10 petaFLOPS at 4-bit precision and more than 5 petaFLOPS at 8-bit. Four-bit numbers sound absurdly coarse until you remember that a model answering a question does not need a weight to fifteen decimal places, only roughly and immediately.
Feeding a chip that fast is the harder trick. Maia 200 carries 216GB of HBM3e, pronounced letter by letter, running at 7 terabytes per second, alongside 272MB of SRAM sitting on the die itself. Picture HBM as a warehouse stacked next to the factory floor and SRAM as the parts bin on the workbench, close enough to reach without standing up. Dedicated data movement engines have one job, which is keeping that parts bin full.
The networking choice is the part engineers will argue about. Rather than a proprietary fabric, Microsoft built a two-tier scale-up network on ordinary Ethernet, with a custom transport layer and a tightly integrated network card. Each accelerator gets 2.8 terabytes per second of dedicated bidirectional scale-up bandwidth, and the company says collective operations stay predictable across clusters of up to 6,144 accelerators. Inside a tray, four Maia chips are wired straight to one another with no switch in the middle.
Now the claims, which deserve a longer look than a product page usually gets. Microsoft says Maia 200 delivers three times the FP4 performance of Amazon's third-generation Trainium and FP8 performance above Google's seventh-generation TPU, and that it runs 30 percent better on performance per dollar than the newest hardware already in the Azure fleet. No benchmark methodology accompanies those figures, and no independent party has published a run of its own. Cross-vendor comparisons shift with the model, the batch size, and software maturity, so the ranking stands as a vendor scorecard until somebody else keeps score.
Scott Guthrie, the executive vice president for cloud and AI who announced the chip, wrote that "AI models were running on Maia 200 silicon within days of first packaged part arrival." The company credits a pre-silicon simulation environment for that, and puts the gap between first silicon and first rack deployment at less than half what comparable programs take. Workloads named for the chip include the latest GPT-5.2 models from OpenAI, Microsoft Foundry, and Microsoft 365 Copilot. The bet is not that Maia outruns Nvidia; it is that enough inference traffic can move onto in-house silicon to change what a token costs.
Deployment starts in Iowa, with a region near Phoenix, Arizona to follow and others after that. Developers can sign up for a preview of the Maia software kit, which bundles PyTorch support, a Triton compiler, a low-level programming language, and a simulator with a cost calculator. That last item says a great deal about what Microsoft thinks the selling point is, and it is not petaFLOPS.
Leave a Reply