TL;DR: A knowledge graph stores facts as a network: entities (people, products, companies) are nodes, and relationships ("works at", "depends on", "acquired") are the edges between them. Where vector search finds text that sounds relevant, a graph lets an AI system follow explicit connections - and answer questions that span several hops.
How it works
The building block is the triple: subject, relationship, object. "Acme acquired Initech." "Initech makes TPS-Pro." "TPS-Pro depends on LibFoo." Each fact is small and unremarkable, but stored as connected nodes and edges they become traversable: ask "which of Acme's products are exposed to LibFoo?" and the answer falls out of walking the edges - acquisition to product to dependency - even though no single document ever states it. That multi-hop question is exactly where plain semantic search struggles, because no chunk resembles the question.
Building the graph used to be the expensive part, done by hand or with brittle extraction rules. Language models changed the economics: an extraction pipeline can now read documents and propose entities and relationships at scale. The output still needs care - deciding that "Acme Corp" and "Acme Corporation" are one node (entity resolution), pruning hallucinated edges, and keeping a schema of allowed relationship types so the graph stays queryable rather than becoming a hairball.
At query time the graph can serve an AI system several ways. The simplest is graph-augmented RAG: retrieve relevant chunks as usual, then also fetch the neighborhood of the entities they mention, giving the model both the prose and the structured facts around it. Richer designs, popularized by approaches like GraphRAG, build community summaries over the graph so the model can answer broad "what are the main themes" questions no single chunk covers. Agents can also query the graph directly as a tool, treating it as a source of ground truth to check claims against.
Graphs are not a replacement for vector retrieval - they are a complement. Vectors excel at "find text about X"; graphs excel at "how is X connected to Y". The cost is upkeep: an extraction pipeline, resolution rules, and a schema all have to be maintained as sources change, which is why graphs earn their keep on connection-heavy domains and are overkill for a simple FAQ bot.
Where it sits in the AI stack
A knowledge graph is a structured data layer that sits beside the vector index, feeding facts into the model's context:
Most real deployments run this lane in parallel with a vector database, merging structured facts and retrieved prose in the final prompt.
Key tools and implementations
-
Neo4j
The most widely used graph database, queried with Cypher and paired with vector search features.
-
Microsoft GraphRAG
An open-source pipeline that builds an LLM-extracted graph plus community summaries for retrieval.
-
Amazon Neptune
A managed graph database supporting both property-graph and RDF models at scale.
-
Memgraph
An in-memory graph database aimed at fast traversals over frequently changing data.
Related entries
- RAG (retrieval augmented generation) Fetching relevant documents at query time so a language model can answer from your data instead of memory alone.
- Vector database A database that stores embeddings and finds nearest neighbors fast, powering semantic search and RAG.
- Hybrid search Running keyword and vector search together and merging results, so exact terms and meaning both count.
- Agent memory Mechanisms that let an AI agent carry information across steps and sessions beyond what fits in the context window.