TL;DR: Choosing a model is a trade among five axes: capability, latency, cost, privacy, and openness. Rank the axes for your specific task, use benchmarks only to build a shortlist, and decide with your own evals on your own data. Then design for change - the ranking of models shifts every few months, and the right answer is one you can swap.
How it works
Start from the task, not the leaderboard. Capability: how hard is the hardest thing the model must do reliably? Complex multi-step reasoning and agentic work justify a frontier model; classification, extraction, and summarization usually do not. Latency: a chat interface needs tokens streaming in well under a second, while a nightly batch job does not care - and smaller models are consistently faster. Cost: pricing is per token, so cost scales with traffic; a model ten times cheaper that succeeds 95 percent as often wins for most high-volume steps. Privacy: if data cannot leave your infrastructure - or your jurisdiction - hosted APIs may be off the table entirely, which turns the choice into a local LLM or self-hosted deployment question. Openness: open weights buy control, fine-tuning freedom, and immunity from provider deprecation, at the price of running or renting the serving yourself.
Few applications need one answer. The emerging default is a portfolio: a frontier model for the hardest steps, a mid-tier model for routine ones, and sometimes a tiny model for classification and routing. That routing pattern - send each request to the cheapest model that handles it - is often the single biggest cost lever available, and it falls out naturally once you stop asking "which model is best" and start asking "which model does this step need."
The process, then: rank the five axes for your task; use benchmarks nearest your task's shape to cut the field to two or three candidates; then run your own evals on your own data, because leaderboard rank predicts your workload poorly. Finally, assume the answer expires. Models improve, prices drop, and providers deprecate, so keep the model behind an abstraction - a gateway, a router, or just a config value - and re-run the evals when the landscape moves. Choosing well once matters less than being cheap to re-choose.
Where it sits in the AI stack
Model choice is the decision that connects your requirements to everything downstream - the provider you integrate, the costs you carry, and the evals that keep the choice honest:
Key tools and implementations
-
Benchmark leaderboards
Public rankings for the shortlisting step - a coarse filter, never the final call.
-
Provider playgrounds
Browser consoles for trying candidate models on real examples before writing any code.
-
Eval frameworks
Tools that score candidate models against your own test set - the evidence the decision should rest on.
-
Gateways and routers
Abstraction layers that put many providers behind one API, making the chosen model cheap to swap.
Related entries
- Model benchmarks Standardized tests that score model capabilities, useful for rough comparison but easy to overfit and game.
- Inference provider A company that runs AI models on its own hardware and sells access through an API.
- Local LLM Running a language model entirely on hardware you control instead of calling a hosted API.
- Open-weights models Models whose trained weights are published for anyone to download, run locally, and fine-tune.
- Evals (AI evaluations) Structured tests that score an AI system's outputs so teams can measure quality and catch regressions.