A $500 training run made a 9B model beat every frontier model at one boring job

A write-up published on Monday by the consultancy Fermisense, and quickly voted up on Hacker News, makes a claim that sounds too good until you read the setup. On a single narrow task, a 9-billion-parameter open-source model, trained for about three and a half days on two rented GPUs for roughly $500, outscored every frontier model they tested, including GPT-5.6 Sol, Gemini 3.1 Pro and Claude Fable 5.

The task is catalogue review: looking at an e-commerce listing with its photo, title and description, filing it in the right category out of about 13,000, extracting the attributes that power search and filters, and deciding whether it breaks policy. Fermisense rebuilt that workflow as what they call a digital twin, a simulated version of the real job with the same tools and the same stakes, using 177,767 review episodes built from the public Amazon Berkeley Objects dataset with policy violations deliberately planted in. Every episode has a known right answer, so a scorer can grade each decision, penalising missed violations seven times more heavily than false alarms. That scorer is what makes reinforcement learning possible: the model tries, gets a number back, and adjusts.

The results. The best frontier configuration, even with 2,800 characters of carefully tuned prompt instructions, reached 76.9 percent of the achievable score. The trained 9B model reached 87.3 percent, up from 64.2 percent before training. It crossed the frontier band after about 250 training steps, roughly one day. Cost per 1,000 listings: about $0.50, against $19 for the cheapest frontier option and $172 for the most expensive. One detail deserves its own sentence, because it is the clearest explanation of the whole result: those optimised prompt instructions inflated input-token bills by 28 to 55 percent on every single call, forever. Prompted knowledge is rented per call. Trained knowledge is bought once and lives in the weights.

Why does a small model win here at all? Because the task is not hard, it is specific. A frontier model starts every listing from zero, knowing nothing about this shop’s taxonomy or which attribute values count as supported, and has to reconstruct all of it from the prompt each time. Fermisense are frank about where the trick fails: the workflow has to run often enough for cost to matter, and its outcome has to be checkable by a rule or rubric. If a decision can only be argued about, it cannot be scored, and if it cannot be scored, there is nothing for the model to practise against. Also worth naming plainly: Fermisense sells consulting on exactly this, and the benchmark is their own. The eight other deployments they cite, from Bridgewater to Intercom to LinkedIn, are each company’s self-reported number.

What this means for you: If AI feels like something you rent from three American companies, this is the counter-example worth knowing about, and the direction of travel is towards cheap specialists rather than one giant brain for everything. Nothing here is a weekend project, though: it took a simulated environment, a scoring rubric and real engineering. If you run a business with a repetitive decision that happens thousands of times a day and has a checkable right answer, the honest question this raises is not whether to fine-tune but whether you are currently paying frontier prices for work that does not need frontier intelligence.

Sources

• The Rise of Intelligence Ownership, Fermisense
• Discussion thread, Hacker News
• Learning to Replicate Expert Judgment in Financial Tasks, Thinking Machines Lab
• Leveraging multimodal LLMs at scale, Shopify Engineering