← Glossary Term
LiveCodeBench
A coding benchmark that keeps adding fresh problems so models cannot have memorised the answers during training.
Older coding tests have a contamination problem: the problems and their solutions are on the public internet, so a model may simply have read them. LiveCodeBench addresses this by continuously collecting new competitive programming problems and only scoring a model on problems published after its training cutoff.
That makes the scores harder to game and easier to compare across releases, which is why labs quote it. It still measures a narrow slice of programming, self-contained puzzle solving, and says little about whether a model can work in a large existing codebase.
Mentioned in