Hybrid Search
Why it matters
Pure vector search, which many RAG tutorials use by default, struggles with exact matches: product codes, error messages, function names, people's names, acronyms and rare jargon. Ask for "error E-4012" and an embedding model might return documents about other errors because they're "semantically similar". Keyword search nails that, but misses synonyms and paraphrases: a search for "how to cancel my plan" might miss a page titled "Ending your subscription".
Adding keyword search to a vector-only RAG system is one of the cheapest and most reliable quality wins available. Most vector databases and search engines (Elasticsearch, OpenSearch, Weaviate, Qdrant, pgvector with full-text search and others) support it.
Deep Dive
There are two main ways to search text. Keyword search (also called lexical or sparse search) looks for documents containing the words in your query, with BM25 being the standard scoring method for decades. Vector search (semantic or dense search) embeds the query and documents and finds those closest in meaning, even if they share no words.
Each one fails where the other succeeds. Hybrid search runs both and merges the results.
How BM25 Works, Briefly
BM25 scores a document higher when it contains the query's words often, when those words are rare across the whole collection, and lower for very long documents where word matches are less meaningful. It needs no training and no GPU, and it's hard to beat on queries with specific terms.
Merging the Two Result Lists
The two searches produce scores on different scales, so you can't just add them. Common approaches:
- Reciprocal Rank Fusion (RRF): ignore the raw scores and combine by rank position. A document's fused score is the sum of 1 / (k + rank) across both lists, with k often set to 60. Simple, robust and the most common default.
- Weighted score fusion: normalize both scores and blend them with a weight (for example 70% vector, 30% keyword). More tunable, but needs care to normalize well.
Add a Reranker
A typical high-quality pipeline is: keyword and vector search each fetch, say, 50 candidates; merge them with RRF; then a reranker model reads the query and each candidate together and re-sorts them, keeping the top 5 to 10 for the LLM. Hybrid search maximizes the chance the right chunk is somewhere in the candidates, and the reranker makes sure it lands at the top.
Learned Sparse Search
Between the two worlds are learned sparse models such as SPLADE, which use a neural network to decide which words (including related words not in the text) matter for each document. They keep the exact-match strengths of keyword search while adding some understanding of meaning.
When to Use It
If your users search for names, codes, technical terms or anything they'd put in quotes, use hybrid search. Measure on real queries: for some purely conversational data, vector search alone is fine, but hybrid rarely hurts.