ARC-AGI-3
A puzzle benchmark designed to test whether a model can work out rules it has never encountered before.
ARC-AGI-3 is the third generation of a test built by the ARC Prize foundation, and its whole point is to be un-memorisable. The puzzles are simple grid problems where the rule has to be inferred from a couple of examples, chosen so that a model cannot have seen anything like them in training. Humans find them fairly easy. Models, for a long time, did not.
Scores have climbed sharply during 2026, which is genuinely interesting, but the numbers come with an asterisk. The same model has scored 7.8 percent and 38.3 percent on the same puzzles depending purely on the harness used to run it. That is why ARC Prize insists on a standard test setup, and why labs keep arguing about whether that setup disadvantages them. Treat ARC-AGI-3 scores as a real signal of progress and a poor way to rank two specific models against each other.