Category
Knowing your AI actually works
A demo that looks great tells you almost nothing. This layer is how teams find out whether an AI system really works: evals that score outputs against fixed test sets, judge models that grade at scale, traces that show what each request actually did, red teams that attack the system before strangers do, and cost tracking that keeps the token bill honest.
Entries in this category
- Evals (AI evaluations) Structured tests that score an AI system's outputs so teams can measure quality and catch regressions. Read the entry
- LLM observability Collecting traces, metrics, and quality signals from a live AI system so problems surface before users complain. Read the entry
- LLM-as-judge Using a language model to score another model's outputs against a rubric instead of human review. Read the entry
- Prompt testing Running prompt changes against a fixed set of test cases so an edit cannot silently degrade quality. Read the entry
- Red teaming Deliberately attacking your own AI system to find jailbreaks and harmful failures before attackers do. Read the entry
- Token cost tracking Counting tokens per request and pricing them per model so AI spend is a metric, not a surprise bill. Read the entry
- Tracing Recording every step of an AI request as a tree of spans so you can see exactly what the system did. Read the entry
Where this layer fits
Evaluation wraps around everything else in the stack. It is how you compare models on your own data instead of leaderboards, how you tell whether an agent is actually completing tasks, and how the risks cataloged under safety get tested instead of just worried about.
Not sure where to start? Evals is the entry the rest of this category builds on. Or browse everything at once in the A-Z index.