Elo

Elo was invented to rank chess players. Everyone starts with a number, and it moves up or down depending on who you beat and how strong they were. Beating a much better opponent moves your rating a lot; beating a much weaker one barely moves it.

AI leaderboards use the same maths, comparing model outputs pair by pair, either judged by people or by another model. That makes Elo useful for questions with no single right answer, like which of two drafts is better written. The number only means something relative to the other models in the same pool, so an Elo score from one leaderboard cannot be compared with one from another.