TL;DR: Benchmarks are standardized test sets that score models on knowledge, reasoning, coding, and other skills, so different models can be compared on the same questions. They are useful for building a shortlist and nearly useless as a final answer: contamination, saturation, and teaching-to-the-test all inflate scores, and no benchmark measures your task. Your own evals are the ground truth.
How it works
A benchmark is a fixed set of problems plus a scoring rule. Knowledge and reasoning suites ask exam-style questions across many subjects; math benchmarks pose competition problems; coding benchmarks hand the model real programming tasks and verify the output by running unit tests. Because every model faces the same questions under the same rules, the scores are comparable, and leaderboards aggregate them into rankings. A separate approach skips fixed questions entirely: arena-style leaderboards show people two anonymous model answers and rank models by which one humans prefer.
The limits are structural, not incidental. Contamination: models train on internet-scale data, and benchmark questions leak into it, so a high score may just mean the model has seen the answers. Saturation: once top models cluster near the ceiling of a test, it stops distinguishing them, and the field moves to a harder one - which is why the named benchmarks change every couple of years while the categories persist. Teaching to the test: labs know which numbers appear in launch posts, and training can chase those numbers in ways that do not generalize. Arena rankings dodge contamination but inherit human quirks - raters reward confident, lengthy, nicely formatted answers, which is not the same thing as correct ones.
The deepest limit is relevance. A benchmark measures performance on its questions, not on your workload - a model can lead a general leaderboard and still lose to a cheaper rival at extracting fields from your invoices. The practical use, laid out in choosing a model, is as a coarse filter: use benchmarks nearest your task's shape to pick a shortlist, then decide with your own evals on your own data. Treat small score gaps as noise, and treat any single number as marketing until you have reproduced it.
Where it sits in the AI stack
Benchmarks sit between a model's release and your adoption of it - the public filter that feeds a shortlist, before private evals make the real call:
Key tools and implementations
-
Academic suites
Multi-subject exam sets testing knowledge and reasoning - the scores quoted in every model launch post.
-
Coding benchmarks
Real programming tasks graded by running unit tests, the closest thing to objective scoring.
-
Preference arenas
Blind side-by-side votes from real users, aggregated into rankings that resist contamination but reward style.
-
Eval harnesses
Open-source frameworks that run published benchmarks against any model, so you can reproduce a claimed score.
Related entries
- Choosing a model A framework for weighing capability, latency, cost, privacy, and openness when picking a model for a task.
- Evals (AI evaluations) Structured tests that score an AI system's outputs so teams can measure quality and catch regressions.
- Prompt testing Running prompt changes against a fixed set of test cases so an edit cannot silently degrade quality.
- Reasoning models Models trained to work through problems step by step before answering, trading time and tokens for accuracy.