TL;DR: Semantic search ranks results by meaning rather than by matching words. A query like "how do I get my money back" finds the page titled "Refund policy" even though the two share no vocabulary. It works by converting queries and documents into embeddings and returning whichever documents sit closest in that numeric space.
How it works
Traditional keyword search treats a document as a bag of words and scores it by how often the query's terms appear. That breaks the moment people phrase things differently: a support article about "canceling a subscription" is invisible to someone searching "stop billing me". Semantic search sidesteps vocabulary entirely. An embedding model converts each document into a vector - a long list of numbers that captures what the text is about - and stores those vectors in a vector database.
At search time, the same model embeds the user's query, and the database returns the documents whose vectors are nearest to the query's vector, usually measured by cosine similarity. Because the embedding model was trained on enormous amounts of text, it has learned that "refund", "money back", and "reimbursement" belong in the same neighborhood - so all three phrasings land near the same documents without anyone maintaining a synonym list.
The catch is that meaning-based matching is fuzzy by design. Semantic search can miss things keyword search nails: exact product codes, error strings, people's names, legal citations. A query for "ERR_CONN_RESET" wants that literal string, not documents about networking in general. That is why production systems rarely run semantic search alone - most pair it with keyword retrieval in a hybrid search setup that keeps the strengths of both.
Quality depends almost entirely on the embedding model. A model trained mostly on English web text will do poorly on medical German or on source code; picking one suited to your domain matters more than any database setting. Chunk boundaries matter too, since a vector can only represent what the chunk actually contains - a chunk that splits a sentence in half embeds a half-thought.
Where it sits in the AI stack
Semantic search is the query-time half of retrieval. It consumes the embeddings and index that ingestion produced, and its ranked results feed whatever comes next - a search results page, or the context window of a RAG system:
In a RAG pipeline it is the retrieval step by another name; in a product it can simply be a better search box. The same index serves both.
Key tools and implementations
-
pgvector
Adds similarity search to Postgres, so semantic queries run next to your existing tables.
-
Elasticsearch and OpenSearch
The classic keyword engines, both of which now index dense vectors alongside text fields.
-
Qdrant
An open-source vector engine with strong metadata filtering for scoped semantic queries.
-
Sentence-transformers
The open-source library most teams reach for to embed queries and documents locally.
Related entries
- Hybrid search Running keyword and vector search together and merging results, so exact terms and meaning both count.
- Vector database A database that stores embeddings and finds nearest neighbors fast, powering semantic search and RAG.
- Embedding models A model that converts text into a vector of numbers so similar meanings land near each other.
- Embedding A list of numbers capturing a piece of content's meaning, so similar things sit close together in vector space.