Microsoft Built a Chip to Take On Nvidia. The Argument It's Making Is About Megawatts.

Every headline about Microsoft's new chip says the same thing. Microsoft built custom silicon to loosen Nvidia's grip on AI computing, and this third generation is finally good enough to matter.

That framing is not wrong. It is the easy half. The number Microsoft led with when it unveiled Maia 300 at the OCP Global Summit on October 12 was not a win against an Nvidia GPU. It was 40% more tokens per megawatt than "leading GPU-based systems." Read that with the emphasis on the unit. Microsoft is not really promising a faster chip. It is promising more useful AI work out of every megawatt it can get its hands on.

The claim, in the company's own words

Scott Guthrie, the executive vice president who runs Microsoft's cloud and AI group, put the strategic turn bluntly in the announcement post: "We will scale our Azure Maia family to multi-gigawatt deployments starting in 2027, making Maia the primary inference system for Microsoft products and services."

Satya Nadella called it "our biggest bet yet on our own silicon." Mustafa Suleyman, who runs Microsoft AI, framed it as a question of control: "We have to work across the full stack, all the way down to the silicon. At that scale we need a platform that we own, and that we co-design with our models."

The silicon is a serious piece of engineering. Each Maia 300 accelerator carries eight HBM4 stacks for 288 GB of memory at 18 TB/s inside a 950-watt package. Its predecessor, Maia 200, shipped 216 GB of HBM3E at 7.1 TB/s in 750 watts. Microsoft built the main compute chiplets on TSMC's N2P process and moved the memory-controller and I/O work onto separate dies. Reuters reported that some of those support chiplets use a cheaper 3nm process, a cost decision Nvidia's monolithic approach does not make.

The rack is where the argument turns concrete. A Maia 300 rack holds 96 accelerators inside a 167-kilowatt envelope, double the 48 accelerators and 64 kilowatts of a Maia 200 rack. Across that rack, Microsoft claims up to seven times the token throughput of the previous generation and more than 2.5 times the performance per watt. Those chips can be grouped into a superpod of 12,288 accelerators linked by a custom Ethernet fabric, so a job can span racks without routing through a host processor.

Why tokens per megawatt is the right number, and the hard one

Power is the binding constraint on AI expansion, and the industry knows it. A new data centre takes years to build, and its power envelope is largely fixed once the concrete is poured. So the useful question stops being how fast a chip runs at its peak and becomes how much work you can pull from the electricity you already have. The analysts at Moor Insights make the fuel-economy comparison: horsepower tells you what an engine can do, miles per gallon tells you what it costs to drive.

Rani Borkar, who runs Azure hardware systems and infrastructure, told Reuters the design is "basically delivering more intelligence for every watt, every dollar, and every byte of memory by optimizing across the entire system." The economics are not subtle. Microsoft has said it expects to spend more than $175 billion in capital expenditure this calendar year, and Reuters reports it plans to deploy potentially more than a million Maia 300 chips next year.

There is a comparison in the Reuters report that Microsoft's marketing skips. A Maia 300 rack draws about 167 kilowatts for 96 chips. Nvidia's current GB300 server rack draws about 142 kilowatts for 72 GPUs. Microsoft is putting more compute into more power and arguing its efficiency per unit of that power is better. That is the whole bet in one line.

The part Microsoft did not publish

The 40% figure has a hole in it. Microsoft did not name the GPU it compared against, the model it ran, or the serving configuration. And because Maia runs only inside Microsoft's own fleet, nobody outside the company can test it. "Nobody has tested them outside of Microsoft yet," Patrick Moorhead of Moor Insights and Strategy told CNBC.

That is not a small caveat. SiliconANGLE noted that Maia 300's 288 GB matches Nvidia's Rubin accelerator on capacity but trails it by roughly 20% on memory bandwidth, the metric that matters most for the decode-heavy inference Microsoft says it is building for. StorageReview made the point in plainer terms: the published specifications support the design, but the headline efficiency claim rests on a comparison the reader cannot check.

Microsoft was also careful not to overcommit on timing. Alistair Speirs, a general manager on the team, declined to put a date on the multi-gigawatt build-out, telling CNBC the company reserves "a little bit of optionality there, because this is an industry that's changing very fast." Production deployments begin in 2027. Evercore ISI analysts estimate Maia will represent about 10% of Microsoft's AI chip spending by the first quarter of 2027. A decade-long margin story, as finance chief Amy Hood has described the internal-silicon effort, starts at a tenth.

The tension the announcement buried

Here is the awkward part. Maia 300 is built around HBM4, and advanced memory has become the tighter input in the very quarter Microsoft is betting on it. This week Korean memory stocks fell sharply on reports about new Chinese DRAM capacity, with Samsung and SK hynix among the worst hit, according to AP and Yonhap reporting. The constraint Maia 300 is designed to relieve, power, now sits next to a second constraint, memory supply, that no dataflow architecture fixes.

Microsoft is also not walking away from Nvidia. Speirs said the company still partners with Nvidia and AMD for training models, and GeekWire notes Microsoft will be among the first cloud providers to deploy Nvidia's Vera Rubin and AMD's Helios systems. Maia is a tool for the inference workloads Microsoft owns end to end, not a replacement for the training fleet. And unlike Google's TPUs or Amazon's Trainium, Maia cannot be rented by the hour by anyone else. Enterprises will meet this chip the way they meet most silicon: abstracted away behind Copilot, GitHub Copilot and Microsoft Foundry.

What it means for the people buying AI

The interesting shift is not that a hyperscaler built another accelerator. AWS has Trainium, Google has its TPUs, Meta has MTIA, and the list keeps growing. The shift is the metric. Microsoft is telling the market it will compete on tokens per megawatt and tokens per dollar, and that it will size its hardware around the workloads it can see in its own telemetry rather than around a benchmark anyone can run.

For enterprise IT, the practical lesson travels. If you run AI infrastructure of your own, power is your constraint too, and most corporate data centres cannot support the densest racks on the market without a major electrical upgrade. Measuring your own stack in tokens per watt, rather than peak flops, tells you far more about what you are actually spending.

The harder question is what a 10% share of spending by early 2027 really signals. Microsoft has made the strategic commitment public, and the silicon backs it up on paper. Whether the efficiency claims survive contact with a production fleet is something only Microsoft will ever be in a position to know.

Planning around AI infrastructure that has to run inside a fixed power envelope, with a supply chain repricing memory faster than most hardware roadmaps can absorb, is the kind of problem that does not fit neatly into a vendor data sheet. At DMC we work with hardware and platform teams modelling exactly those constraints: power-to-token economics, memory sourcing under shortage, and the build-versus-rent decision as custom silicon and merchant GPUs diverge. If your AI roadmap needs a stress test before the next budget cycle, let's talk.