Hybrid search RAG combines keyword retrieval with vector retrieval so a RAG system can find both exact terms and semantically related passages. In practice, the application runs lexical and vector search against the same authorized knowledge base, merges the ranked results, optionally reranks the shortlist, and sends the best evidence to the language model.
This matters because enterprise queries mix precise identifiers with natural language. A support engineer may search for ERR_AUTH_706; a policy user may ask, “How do I regain access after my account is locked?” One retrieval method rarely handles both cases equally well.
For the wider architecture around ingestion, retrieval, permissions, and grounded generation, see LuMay’s Enterprise AI Architecture Patterns and RAG Development Services.
Quick Answer: How Do You Combine Keyword and Vector Search?
A practical enterprise pattern is:
- Preserve the user’s exact terms and identity context.
- Build one set of metadata and permission filters.
- Run BM25 keyword retrieval and vector retrieval in parallel.
- Merge candidates with score normalization, weighted fusion, or Reciprocal Rank Fusion (RRF).
- Deduplicate by a shared chunk ID.
- Optionally rerank the fused shortlist.
- Assemble permitted passages with source metadata.
- Generate an answer with citations and log the retrieval trace.
Keyword Search: Precision on Exact Terms
Keyword search, commonly implemented with an inverted index and BM25-style ranking, rewards textual overlap between the query and the indexed content.
It is especially useful for:
- Error codes and log strings
- SKUs, model numbers, invoice IDs, and ticket numbers
- Policy IDs, clause numbers, and regulatory citations
- Names, dates, acronyms, and product terminology
- Queries where an exact phrase matters more than a related concept
For example, a query for CVE-2026-18421 XG-400 contains two high-value identifiers. Lexical retrieval can match those strings directly instead of hoping an embedding model preserves their distinctions.
The limitation is vocabulary mismatch. A user may ask about “hotel costs after a cancelled flight” while the relevant policy says “accommodation due to travel disruption.” Keyword search can miss or under-rank the right passage when the wording changes substantially.
Vector Search: Recall Across Meaning and Paraphrases
Vector search converts the query and document chunks into embeddings and retrieves passages that are close in semantic space.
It performs well for:
- Natural-language questions
- Paraphrases and synonyms
- Conceptual search across policies, manuals, and tickets
- Queries where users do not know the source document’s exact terminology
- Multilingual retrieval when the embedding model supports the required languages
A question such as “How can an employee regain access when their account is locked?” may still retrieve a document titled “Identity Reset and Credential Recovery Procedure,” even though the wording barely overlaps.
Vector-only retrieval has a different weakness: exact tokens can blur. Product X2 and X20, Policy 401 and 410, or two similar error codes may appear semantically close even when the distinction is operationally critical.
Keyword vs Vector vs Hybrid Search
| Dimension | Keyword Search | Vector Search | Hybrid Search |
|---|---|---|---|
| Core signal | Exact term overlap | Semantic similarity | Both signals |
| Typical method | BM25 / sparse text retrieval | Dense embeddings + ANN | Parallel retrieval + fusion |
| Best at | IDs, names, codes, clauses, dates | Paraphrases, concepts, natural questions | Mixed enterprise queries |
| Common miss | Different wording for the same concept | Exact identifiers and rare strings | Poor fusion, filtering, or tuning |
| Main tuning levers | Analyzers, synonyms, field boosts | Embedding model, chunking, ANN settings | Candidate depth, fusion, weights, filters |
Hybrid search gives retrieval two complementary ways to discover evidence; evaluation determines how much each signal should influence the ranking.
A Practical Enterprise Hybrid RAG Architecture
1. Prepare the query without destroying exact terms
Normalize obvious formatting, resolve conversational follow-ups, and extract structured fields such as dates, products, regions, or document types. Preserve identifiers such as FIN-POL-428, because rewriting them into a semantic paraphrase can weaken lexical retrieval.
2. Build metadata and permission filters
Create one filter model for both retrieval paths. Useful fields include tenant, department, region, document type, effective date, version, confidentiality level, and user or group access.
Enterprise RAG should not retrieve unauthorized content and rely on the LLM to ignore it. LuMay’s Data Readiness for Generative AI guide likewise treats usage rights and access constraints as data-readiness requirements.
3. Run lexical and vector retrieval in parallel
Send the textual query to the keyword index and the query embedding to the vector index. Each path returns its own top-K candidate list.
Microsoft’s Azure AI Search hybrid query documentation describes full-text and vector queries executing in parallel before their results are merged.
4. Fuse the candidate lists
Keyword and vector scores usually live on different scales, so raw addition is unsafe. Use an explicit fusion strategy such as normalization, weighted score fusion, or RRF.
5. Deduplicate and preserve lineage
Use a shared chunk ID across both paths, deduplicate repeated hits, and preserve document ID, version, source URL, and permission metadata for citations and debugging.
6. Optionally rerank the fused shortlist
Fusion and reranking solve different problems. Fusion combines the two retrieval lists. A reranker evaluates the already-fused candidates and changes their order.
A reranker cannot recover a document that neither retriever surfaced, so fix recall first.
7. Assemble context and generate a grounded answer
Select the final authorized passages, remove redundant chunks, respect the prompt’s token budget, and attach citation metadata. The model should answer from that evidence and abstain when the evidence is insufficient.
Three Ways to Merge Keyword and Vector Results
1. Score Normalization
Score normalization maps each retriever’s scores to a comparable range before they are combined. A simplified min-max form is:
normalized_score = (score - min_score) / (max_score - min_score)OpenSearch’s normalization processor supports min-max, L2, and z-score normalization before combination.
When it helps: when score magnitude contains useful information and you want fine control.
Watch for: outliers, small result windows, and score distributions that vary substantially by query.
2. Weighted Score Fusion
After normalization, apply explicit weights:
hybrid_score = alpha × vector_score + (1 - alpha) × keyword_scoreA technical-support workload containing model numbers and error codes may justify stronger lexical weighting. A natural-language policy assistant may justify more semantic weight.
Weaviate’s hybrid search documentation exposes an alpha parameter for adjusting the contribution of keyword and vector results.
Tune weights on representative queries rather than copying another corpus's default.
3. Reciprocal Rank Fusion
RRF ignores raw scores and combines results using rank position:
RRF(d) = sum(1 / (k + rank(d)))Because it uses ranks instead of BM25 and vector score magnitudes, RRF avoids the score-calibration problem. Elasticsearch documents RRF for combining result sets with different relevance indicators, making it a strong baseline before score calibration.
Worked Example: Why Hybrid Retrieval Helps
Assume the knowledge base contains:
- D1: “ACME Router XG-400 requires firmware patch 7.3.14 for CVE-2026-18421.”
- D2: “Network appliances affected by the October security vulnerability should follow the emergency remediation procedure.”
- D3: “The XG-400 installation manual covers rack mounting and power requirements.”
Query A: CVE-2026-18421 XG-400
Keyword retrieval has a strong advantage because the CVE and model number identify D1 precisely. Vector retrieval may return broader cybersecurity content.
Best starting point: keyword-heavy hybrid retrieval.
Query B: “What should we do with network devices affected by the latest security issue?”
D2 expresses the concept without matching all the query words. Vector retrieval has a clear advantage, while keyword retrieval can still contribute candidates around “network” and “security.”
Best starting point: vector-heavy or balanced hybrid retrieval.
Query C: “How should XG-400 devices be remediated after the security vulnerability?”
The query contains both a precise model number and a semantic intent. Keyword retrieval may identify D1 and D3; vector retrieval may identify D1 and D2. Fusion can promote D1 because both retrievers support it while still keeping D2 as useful procedural evidence.
Hybrid retrieval combines lexical precision with semantic recall.
Enterprise Considerations Beyond the Retrieval Algorithm
Metadata and access controls
Use deterministic filters for region, document status, version, product family, and user authorization. Apply equivalent access rules to both retrieval paths. A common failure is protecting the lexical index while the vector path can still surface restricted chunks.
Chunking and index consistency
Hybrid search does not fix poor content preparation. Preserve headings, definitions, tables, exceptions, dates, and section boundaries during chunking. Record the same chunk identifier in both lexical and vector representations so fusion and deduplication are reliable.
Latency and cost
Run lexical and vector retrieval in parallel. Retrieve a controlled candidate window, fuse it, and reserve more expensive reranking for the shortlist. Increasing top-K can improve recall but also increases latency and downstream cost.
Evaluation
Build a labeled evaluation set containing:
- Exact-identifier queries
- Paraphrase queries
- Mixed identifier + concept queries
- Date- or version-sensitive questions
- Permission-sensitive questions
- Ambiguous questions where the right outcome may be abstention
Compare keyword-only, vector-only, and hybrid configurations on the same set. Useful retrieval metrics include Recall@K, Precision@K, MRR, and nDCG, depending on your evaluation design.
LuMay’s LLM Evaluation Metrics guide covers downstream groundedness, citation accuracy, latency, and task-quality measures.
Observability
Log enough data to reconstruct why a passage was selected:
- Original and normalized query
- Applied metadata and permission filters
- Keyword candidate IDs and ranks
- Vector candidate IDs and ranks
- Fusion method and weights
- Fused order and reranker score, if used
- Final chunks sent to the model
- Retrieval latency, errors, citations, and user feedback
This trace makes retrieval failures diagnosable instead of opaque.
Fusion vs Reranking
| Stage | Purpose | Input | Output |
|---|---|---|---|
| Retrieval fusion | Combine lexical and semantic evidence | Two ranked candidate lists | One hybrid candidate list |
| Reranking | Improve ordering of known candidates | Fused shortlist | Reordered shortlist |
Fusion protects candidate recall. Reranking improves precision among candidates already found. Keep them separate in both architecture and evaluation.
When Should an Enterprise Use Hybrid Search RAG?
Hybrid retrieval is most useful when a corpus mixes exact terminology with natural language, such as technical support, policy search, legal knowledge, product documentation, internal service desks, and engineering knowledge bases. Keyword-only may still suit structured lookups; vector-only may suit narrowly conceptual workloads.
Build Hybrid Retrieval With LuMay AI
LuMay’s RAG Development Services include semantic, keyword, and hybrid retrieval, metadata filtering, query rewriting, reranking, citations, and RAG evaluation. The broader architecture can also incorporate permission-aware retrieval, enterprise integrations, monitoring, and production deployment.
If you are designing an enterprise knowledge assistant or modernizing an existing RAG stack, bring a sample of your documents, permission model, and representative queries. Book a LuMay AI discussion to review the retrieval architecture and evaluation plan.
