SWE-Bench

SWE-Bench is a benchmark that measures whether an AI can resolve real issues from real software projects, not toy puzzles. Scores on it are the most-quoted numbers in any coding-model announcement, and the harder SWE-Bench Pro variant arrived once models began saturating the original.

Benchmarks are imperfect, models can be tuned to the test, but SWE-Bench remains the quickest honest signal of coding ability.