NVIDIA's Ian Buck Pitches Tokens-Per-Watt as the New AI Factory Metric at Packed AI Infra Summit

NVIDIA vice president Ian Buck took the stage Tuesday at the AI Infra Summit in Santa Clara to make a case that the industry's efficiency scoreboard needs rewriting. Buck, who leads hyperscale and high-performance computing at the chipmaker, framed tokens per watt as the yardstick that matters for operators running large-scale inference, rather than peak throughput alone. The talk landed at a conference that has grown fast enough to become the infrastructure crowd's own festival: more than 8,000 attendees this year, roughly double the 3,500 who showed up a year ago.

The pitch rests on the economics of so-called AI factories, the purpose-built clusters where models are trained and then served to paying customers. In that setting, every wasted joule shows up on the power bill and in the queue of requests waiting on capacity. Buck's argument is that optimizing the full stack around tokens produced per watt of power consumed can stretch the same electrical envelope further than simply adding accelerators. That reframes efficiency as a systems problem spanning silicon, networking, scheduling and the software that decides how work gets batched.

NVIDIA's Vera Rubin platform and its DSX reference designs sat at the center of the presentation, positioned as the hardware answer to that metric. Vera Rubin is the company's next-generation data center architecture, following the Blackwell generation, and DSX covers the rack-scale integration work that turns chips into deployable factory modules. The company's message to operators is that the gains come from co-designing these layers together, not from swapping in faster parts and hoping the rest keeps up.

The timing is not accidental. Data center power has become the binding constraint on AI expansion in many markets, with utilities, landlords and local governments increasingly scrutinizing how much electricity new campuses will draw. Operators signing multi-year capacity deals want to know how many tokens they can serve per megawatt, because that number sets their unit economics. A vendor that can credibly move that ratio gets an advantage in procurement conversations that increasingly involve power contracts as much as compute.

The conference's growth from 3,500 to over 8,000 attendees in a single year tells its own story about where attention has shifted. Infrastructure, once the unglamorous plumbing beneath model launches, is now where the hardest constraints and largest budgets live. A keynote slot at an event that size is itself a signal that NVIDIA wants to define the terms of that conversation before rivals and customers settle on their own benchmarks.

What remains open is whether tokens per watt becomes a shared industry measure or stays a vendor talking point. Independent benchmarking of these claims is difficult, since results depend heavily on model size, batch size, latency targets and the mix of workloads a cluster actually serves. The next test will be whether customers and competitors adopt the same unit when they publish their own numbers, or whether each camp keeps its own scoreboard.