IBM's Granite 4.2 Is Free, Small Enough to Run Locally, and Built to Think First

IBM has released Granite 4.2, a family of open language models in 3 billion, 8 billion, and 30 billion parameter sizes, under an Apache 2.0 licence. That licence matters more than the benchmark chart: it means you can download the models, adapt them, and run them in a product without asking permission or paying a fee. They are on Hugging Face, Ollama, LM Studio, GitHub, and OpenRouter as of Tuesday.

The headline change is native reasoning. These models think step by step before acting, which in practice means planning a sequence of tool calls rather than firing them off and hoping. Parameters, roughly, are the internal numbers a model learns during training, and the count is a decent proxy for how much hardware you need: a 3B model runs comfortably on a laptop, an 8B model wants a reasonable graphics card, and 30B is a workstation or a small server.

Getting there took training changes rather than more scale. IBM added a multi stage reinforcement learning regimen, which is training where the model is rewarded for good outcomes rather than shown correct answers, starting with a foundational stage across maths, science, coding, and tool calling. The 8B and 30B models then went through an “agentic RL” phase on enterprise style tasks: software engineering, terminal work, search driven workflows. The models also trained on a trillion tokens of synthetic code from IBM’s CodeAlchemy pipeline, and they ship with a speculative decoding layer, a trick that guesses several words ahead and checks them in one pass, so text comes out faster.

Quietly shipped alongside are two speech models, Granite Speech 5.0 Turbo CTC and a non commercial variant, at just 470 million parameters. They have no language model inside them at all, which is what keeps them tiny. IBM reports a throughput score of roughly 12,600 on a single H200 GPU against about 6,000 for the current speed leaders on Hugging Face’s Open ASR leaderboard, and says the model transcribes three hours of recorded voice in about a second.

What’s actually going on here: IBM is not trying to beat GPT-5.6 or Claude. It is aiming at the boring middle of the market, where a company wants an agent that reliably does the same multi step task ten thousand times, on its own servers, without a per token bill or its data leaving the building. That is a real and underserved need, and open weights plus small sizes is a sensible answer to it. The competitive charts IBM publishes are its own, against similarly sized models, so read them as a claim rather than a verdict.

What this means for you: if you have wanted to try a capable AI model on your own computer, the 3B and 8B versions are an unusually easy starting point. ollama run granite4.2 and you are going, offline, with nothing sent anywhere. Do not expect frontier quality; expect something useful for summarising, drafting, and simple tool use that costs nothing and cannot be discontinued out from under you.

Sources

• IBM Research: Granite 4.2 brings native reasoning to enterprise agents
• Technical breakdown on Hugging Face
• Granite 4.2 on Ollama
• The Decoder: IBM drops open-weight Granite 4.2 family under Apache 2.0