OpenAI Built Its Own Chip, and the First Benchmarks Are Good

OpenAI and Broadcom presented Jalapeño at the Hot Chips conference this week, the first processor OpenAI has designed itself. It is an inference chip, meaning it is built for running finished models rather than training new ones, and the first published benchmarks put it ahead of Nvidia’s current racks on the measure that matters most to anyone paying an electricity bill.

The headline figures, from the chip analysis newsletter SemiAnalysis and from OpenAI’s own numbers: Jalapeño draws 700 watts, against 900 to 1,150 watts for Nvidia’s Rubin compute die. On SemiAnalysis’s InferenceX benchmark it delivered 1.5 to 1.9 times more throughput per kilowatt and 1.7 to 3.6 times lower end-to-end latency than Nvidia’s GB200 and GB300 rack systems, tested across open models including DeepSeek R1, GPT-OSS and Kimi K2.5. Design work began in mid-2024, the design went to the factory in November 2025, and it is built on TSMC’s N3P process with HBM4 memory at 15.4TB/s. Two caveats sit in the same reporting. SemiAnalysis argues the fair comparison is Nvidia’s newer Vera Rubin platform rather than Blackwell, since both use HBM4 memory, and on that footing Jalapeño still edges ahead on output tokens per megawatt but comes out roughly even on total cost per token.

What’s actually going on here: inference is where the money goes. Training a model is a one-off expense, but every question a user asks costs power forever after, and at OpenAI’s scale a fifth off the electricity bill is a very large number. A custom chip can be narrower than a general-purpose GPU by design: it only has to run OpenAI’s own models well, so everything that does not serve that goal gets cut. That is the entire reason Google built TPUs and Amazon built Inferentia. The strategic part is not the benchmark, it is the leverage. A company with a credible in-house chip negotiates with Nvidia from a different position, and Anthropic reading the same page is presumably why it just hired the founder of Google’s TPU programme.

What this means for you: nothing you will notice this month, and you cannot buy one. But cheaper inference is the mechanism behind almost every improvement you actually feel: lower API prices, higher free-tier limits, faster answers, and models being allowed to think for longer before replying. If you build on AI APIs, the thing to watch is whether OpenAI passes the efficiency on or keeps it as margin. Also worth grounding: these are vendor and analyst benchmarks on selected models, not independent testing at scale, and Nvidia’s answer ships in the same window.

Sources

• OpenAI and Broadcom unveil LLM-optimized inference chip (OpenAI)
• OpenAI Jalapeño: Better Than Nvidia Blackwell (SemiAnalysis)
• OpenAI’s Jalapeño chip is built for fast inference at scale, benchmarks show (TechCrunch)
• OpenAI’s first custom chip reportedly beats Nvidia’s Blackwell and Rubin (The Decoder)