TL;DR: A system prompt is the set of instructions an application sends to the model before any user message - the user never sees it. It defines the model's role, rules, tone, capabilities, and boundaries for the whole conversation. When two products built on the same model behave completely differently, the system prompt is usually the reason.
How it works
Chat APIs distinguish message roles: system, user, and assistant. The system message arrives first, and models are specifically trained to give it more authority than what follows - it is where the application says "You are a customer support agent for Acme. Answer only from the provided documentation. If unsure, offer to create a ticket. Never discuss competitors or pricing." Every user message afterward is interpreted through that lens. A concrete pair: send "Write a poem about the ocean" to a bare model and you get a poem; send it to a model whose system prompt says it is a banking assistant that stays on topic, and you get a polite redirect to account questions.
In production the system prompt does real engineering work. It carries the persona and tone, the operating rules, dynamic context the app injects (the current date, the signed-in user's name and plan), definitions of available tools for tool use, and output format requirements. It is assembled by code at request time - part template, part configuration - and because models are stateless, it is resent with every single request, silently consuming context window budget and token cost on each turn. Long system prompts are a big reason prompt caching pays off.
Its authority has limits worth understanding. A system prompt is an instruction, not an enforcement mechanism: models follow it strongly but not perfectly, and attackers actively probe that gap. Prompt injection tries to smuggle competing instructions in through user input or retrieved documents; "ignore previous instructions" is the classic opener. Well-known products have had their system prompts extracted verbatim by persistent users, so treat the contents as configuration, not as a secret or a security boundary. Anything that must never happen needs guardrails in code, outside the model.
Writing one is ordinary prompt engineering with higher stakes: be specific, cover the failure cases, keep it as short as it can be while doing the job, and put it under version control and testing like the code it effectively is.
Where it sits in the AI stack
The system prompt is the first thing into the context on every request - the frame the rest of the conversation lands inside:
Key tools and implementations
-
Chat API system role
The dedicated message slot every major provider offers for privileged instructions.
-
Prompt templates
Code that assembles the system prompt per request from static rules and dynamic context.
-
Agent instruction files
Repo-level files like AGENTS.md that feed project rules into a coding agent's system prompt.
-
Published system prompts
Some vendors release their assistants' system prompts - useful worked examples of the craft.
Related entries
- Prompt engineering Designing the instructions, examples, and context you give a model to get reliable, useful output.
- Prompt injection An attack where malicious instructions hidden in content an LLM reads override what the developer told it to do.
- Tool use (function calling) Letting a language model request actions by emitting structured calls that your code executes and whose results feed back in.
- Context window The maximum amount of text, measured in tokens, that a model can consider in a single request.