Google's WikiSkill Lets AI Agents Keep a Wiki of Their Own Mistakes

An AI agent today has the memory of a goldfish. It runs a task, makes the same mistake it made yesterday, finishes, and forgets everything. A new paper from Google Research and Virginia Tech, published on arXiv on August 27, proposes a fix that is refreshingly low-tech: let the agent keep a wiki.

The system is called WikiSkill and has three layers. The Raw Layer stores complete execution traces, every tool call and every result, and never changes. The Wiki Layer distils that raw record into structured notes, documented failure patterns on one side and strategies that worked on the other, growing with each round. The Skill Layer holds the actual instructions the agent follows when it works, packaged as reusable modules called Agent Skills. If a new version of a skill makes performance worse, it can be rolled back. The numbers are substantial: Gemini-3.5-Flash went from 49.5 percent to 68.1 percent accuracy, and Qwen-3.6-27B from 39.4 percent to 63.3 percent. In one comparison a 9B model with WikiSkill outperformed a 27B model without it.

What’s actually going on here: it is worth being precise about what is learning here, because the marketing language around this gets slippery. The model itself does not change. Its weights, the numbers fixed during training, stay exactly as they were. What changes is the written instructions the agent reads before it starts working. This is closer to persistent external memory and automated prompt improvement than to a model that keeps learning after release. That sounds like a downgrade, but it is arguably better for practical use: notes are readable, editable and reversible, whereas retraining a model is expensive, slow and opaque. If you want to know why your agent behaves a certain way, you can open the wiki and read it. That is a debugging story, not just a benchmark story.

What this means for you: the immediate lesson is one you can apply by hand today, no research code required. If you use an AI assistant for a recurring task, keep a short document of what worked and what failed, and paste it in at the start of each session. That is WikiSkill in miniature, and the effect is real. Several assistants already support saved instruction files or skills for exactly this. For anyone building on agents, the finding worth taking seriously is the size comparison: accumulated task knowledge closed a gap that would otherwise have cost you a model three times larger, and running costs scale with size. A fair caveat: these results come from the authors’ own evaluations, and a system that writes its own instructions can also write in a bad habit and then follow it faithfully. The rollback layer exists for a reason.

Sources

• Google’s WikiSkill gives AI agents a persistent memory of past mistakes (The Decoder)
• Google’s WikiSkill Helps a 9B AI Model Outperform a 27B Model (XenoSpectrum)