TL;DR: Agent frameworks are libraries that supply the machinery every agent needs - the loop, tool execution, state, retries, streaming - so developers write behavior instead of plumbing. They range from graph-based orchestration libraries to batteries-included agent SDKs. The category is useful, crowded, and optional: many production agents are a few hundred lines of plain code.
How it works
Strip any agent down and the same plumbing appears: call the model, parse a tool request, execute it, append the result, decide whether to loop, handle the timeout, truncate the transcript when it outgrows the context window. None of it is intellectually hard, but all of it must be correct, and every team used to rebuild it from scratch. A framework packages that plumbing behind an interface where you declare the pieces - a model, a system prompt, a set of tools - and the library runs the loop.
The category splits into a few shapes. Orchestration libraries (LangGraph is the archetype) model an application as an explicit graph of steps and give you fine control over state, branching, and cycles - closer to a workflow engine than a chat wrapper. Role-based frameworks like CrewAI organize work as teams of agents with assigned jobs. Agent SDKs from model providers wrap a vendor's own loop, context management, and built-in tools in a supported package. Minimal toolkits deliberately stay small - a typed model client, a tool decorator, and little else.
What you are actually buying is the undifferentiated 80 percent: streaming output, parallel tool calls, retry-with-backoff, session persistence, human-approval hooks, and increasingly MCP support so external tool servers plug in without custom glue. What you are paying is abstraction cost. A framework's loop is opaque where your own would be transparent; debugging means understanding its internals at exactly the moment something misbehaves; and a fast-moving dependency churns APIs under you.
Choosing is less about rankings than fit. A prototype that must exist by Friday suits a batteries-included SDK. A production workflow with compliance gates and audit trails suits an explicit graph. A team fluent in its own stack often does best with a plain loop and no framework at all - model APIs have absorbed the hardest parts (tool calling, structured output, streaming), so the residual plumbing is thinner than the ecosystem's size suggests. Whatever the choice, keep tool definitions and prompts portable; frameworks age faster than the agents built on them.
Where it sits in the AI stack
A framework is the middleware between your application and the raw model API:
The most battle-tested agent harnesses in daily use are coding agents - worth studying as reference implementations even if you never build software with them.
Key tools and implementations
-
LangGraph
Graph-based orchestration with explicit state and checkpoints, for workflows needing fine control.
-
CrewAI
Role-based teams of agents with assigned tasks, aimed at multi-agent collaboration patterns.
-
Provider agent SDKs
Vendor-supported SDKs wrapping a model's native loop, context handling, and built-in tools.
-
Minimal toolkits
Small typed libraries - or no library at all - for teams that want the loop in their own code.
Related entries
- AI agent A system where a language model plans, calls tools, and loops on results to finish a task with minimal supervision.
- MCP (Model Context Protocol) An open protocol that lets AI applications connect to external tools and data through a standard client-server interface.
- Tool use (function calling) Letting a language model request actions by emitting structured calls that your code executes and whose results feed back in.
- AI coding agent Software that plans and executes multi-step coding tasks - reading files, editing code, and running tests with minimal supervision.