Why AI Skills Actually Help, and Why 100 of Them Are Worse Than 5
A study of 8,135 trial runs finds skills work by giving agents a reliable process, not by adding knowledge. Grow the library from 5 to 100 and the agent stops finding the right one.
Skills have quietly become the standard way to make an AI agent better at something without retraining it. A skill is just a written set of instructions: here are the steps for this task, here is what to check, here are the mistakes people usually make. The agent reads it before starting. Every major agent product now supports some version of this, and until now the evidence for them was simply that agents with skills finished more tasks. Nobody had shown why.
A team from Princeton, UC San Diego, and other institutions ran the experiment. They compared agents with and without a skill on identical tasks across 8,135 trial records, then hand coded what actually differed in the transcripts.
The finding is cleaner than expected. Skills help mainly by giving the agent a reliable procedure to anchor to, which accounted for 65.7 percent of the cases where the skilled agent did better. Supplying the agent with facts it did not know explained just 4.5 percent. In other words, a skill is a checklist, not an encyclopaedia. It steadies which setup steps to run, which tools in which order, what to verify along the way, and that removes a whole class of sloppy execution errors. Against Workflow Memory, an earlier approach, skills scored 6.06 points better in matched comparisons.
Skills also introduce failures of their own. In roughly one case in ten the agent applied a perfectly good playbook mechanically, in a situation it did not fit. And the retrieval problem is stark: grow the library from 5 skills to 100 and the precision with which the agent picks the right one collapses from 29.6 percent to 3.3 percent. Skills that sound similar make the choice harder.
What’s actually going on here: there is a comforting corollary buried in the numbers. The researchers found that invoking the exactly correct skill is neither sufficient nor necessary, because a related skill often supplies enough structure on its own, and downstream success held up even when offline identification looked bad. So the picture is not “the agent is lost”, it is “the agent is fuzzy about which playbook it grabbed, and often that is fine”. Their recommendation is to treat skills as a lifecycle problem, where creating, retrieving, and applying them each need work, rather than assuming that storing more experiences makes an agent smarter.
What this means for you: if you write skills or custom instructions for an AI tool, this is the most useful practical result of the month. Write procedure, not background. The model already knows what a spreadsheet is; what it needs from you is the order of operations and the checks. And resist the urge to build a giant library. A small set of clearly distinct skills beats a sprawling one where half the entries sound alike, because past a certain point the agent simply cannot tell them apart. If you use agent tools without writing anything yourself, the takeaway is milder: skills are real, the gains are modest and specific, and nobody has solved the filing problem yet.
Sources
Source: https://arxiv.org/abs/2608.14036
Anthropic Wants AI Agents to Run Lab Equipment, and Has a Standard for It
The Model Hardware Standard is a common language between AI agents and physical machines. Early testers say connecting six lab instruments took under a week instead of months.
