TL;DR: Evals are repeatable tests for AI systems. Instead of eyeballing a few chat transcripts, you run the system against a fixed set of cases, score every answer against criteria you wrote down, and compare the numbers to the previous run. They are how teams ship prompt and model changes on evidence instead of gut feel.
How it works
Everything starts with a golden dataset: a fixed set of inputs paired with what a good answer looks like. For a support bot, that might be 200 real customer questions, each labeled with the correct resolution and a note about acceptable tone. The set should be small enough to run in minutes and honest enough to represent real traffic - including the awkward edge cases that actually break things, not just the happy path from the demo.
Each output then gets scored. Some checks are mechanical: did the answer cite the right refund policy, is the JSON valid, did it stay under a length limit. Fuzzier qualities - helpfulness, tone, faithfulness to sources - are graded by a human reviewer or, more commonly at scale, by an LLM-as-judge working from a written rubric. A run produces a score per case plus an aggregate per criterion, so "the bot got worse" becomes "faithfulness dropped six points on billing questions."
Evals of this kind are offline: they run before a change ships, against a frozen dataset, so any two runs are directly comparable. The complement is online evaluation - scoring a sample of live production traffic - which catches the problems your dataset never anticipated. Mature teams run both, and treat LLM observability as the online half of the same discipline.
The payoff is regression safety. When a new model version or a reworded prompt drops accuracy from 91 to 78, you find out in CI, not from customers. Public model benchmarks answer "which model is generally capable"; your evals answer the only question that matters - whether your system, on your data, got better or worse.
Two failure modes are worth naming. A dataset that never grows becomes a target the system quietly overfits to, so fold real production failures back into it on a schedule. And a metric nobody trusts gets ignored, so spot-check automated scores against human judgment often enough to keep them honest.
Where it sits in the AI stack
Evals sit beside the application rather than inside it. The harness exercises the same prompts, models, and data your production system uses, scores the results, and gates what ships:
Key tools and implementations
-
promptfoo
Open-source eval runner with declarative test cases and assertions, built to live inside CI pipelines.
-
Braintrust
Eval platform pairing datasets and scoring functions with side-by-side diffs between runs.
-
LangSmith
Dataset management and eval tooling tied into tracing, from the LangChain ecosystem.
-
Ragas
Eval library focused on RAG pipelines - faithfulness, context precision, and answer relevance.
Related entries
- LLM-as-judge Using a language model to score another model's outputs against a rubric instead of human review.
- Prompt testing Running prompt changes against a fixed set of test cases so an edit cannot silently degrade quality.
- LLM observability Collecting traces, metrics, and quality signals from a live AI system so problems surface before users complain.
- Model benchmarks Standardized tests that score model capabilities, useful for rough comparison but easy to overfit and game.