Hybrid search combines lexical retrieval with vector retrieval so a RAG system can find both exact wording and semantically related passages. It is useful when users mix identifiers, names and technical terms with ordinary questions. The design challenge is to combine ranked results while preserving access rules, document identity and enough evidence to answer.
A semantic search result can be related to a question without containing the exact part number it asks about. A lexical result can match every term while missing the intended meaning. Hybrid retrieval offers two routes to candidates; its value must be measured on your own query mix.
What each retrieval route contributes
Microsoft's Azure AI Search overview documents an implementation that runs full-text and vector queries in parallel and merges them using reciprocal rank fusion. That is a documented platform design, not a requirement that every hybrid search engine behaves identically.
| Query pattern | Route to test carefully | Potential failure |
|---|---|---|
| Exact product or incident identifier | Lexical search and dedicated identifier fields | Tokenization splits or changes the identifier |
| Question using different wording from the source | Vector retrieval | A similar concept displaces the needed passage |
| Name plus a conceptual question | Both routes | Correct concept from the wrong entity |
| Request for a restricted document | Shared authorization filters | One route searches a broader scope |
Define a common candidate contract
Use a stable document ID, passage ID, source version, source location and access metadata in both indexes. Record the indexed text and its relationship to the original document. A passage duplicated across two retrieval routes should remain one evidence item after fusion.
Decide how updates propagate. If the text index contains a revised policy while the vector index still represents the previous version, a merged result can mix contradictory evidence. Track index freshness and use version checks or an explicit migration strategy.
For numeric or punctuation-heavy identifiers, test analyzer behavior using actual examples from your corpus. Do not assume a general text analyzer preserves a code as a single exact-match value.
Fuse ranks without pretending scores are interchangeable
BM25 and vector similarity scores describe different scoring systems. Adding their raw values without calibration can make one route dominate for accidental reasons.
Microsoft's RRF documentation explains rank-based fusion: each candidate receives contributions based on its position in the result lists. Platform weighting and configuration options vary, so verify your implementation rather than copying settings across engines.
A practical first experiment compares lexical-only, vector-only and fused retrieval on the same held-out questions. Keep candidate budgets and the final context budget visible. If hybrid retrieves more passages, compare the additional cost with its measured benefit.
Implement and verify in five steps
- Prepare aligned indexes. Confirm identity, versions and text boundaries match across retrieval routes.
- Apply the same eligibility rules. Test tenant, user, matter or department filters on both routes. Relevance scoring must never grant access.
- Retrieve and deduplicate. Preserve route provenance and rank position, then combine matching passage IDs.
- Fuse and assemble context. Keep source links and required surrounding conditions. Add a reranker only if ordering failures remain.
- Measure failure classes. Review missing evidence, wrong entity, stale document, duplicate context and unauthorized results separately.
Worked example: an error code with an unusual description
Illustrative example: a support engineer asks, “Why does ZX-41 stop after a credential change?” One article contains ZX-41 but discusses a different failure. Another uses the phrase “authentication refresh failure” and explains the credential-change procedure.
The lexical route should preserve the exact code match. The semantic route may find the procedure despite the different wording. A reviewer determines whether the combined passages support the specific diagnosis; the system should not infer that every credential-related error has the same cause.
Add a negative control with a similar code, ZX-14. The answer must not cite a ZX-41 procedure as if it were verified for ZX-14. This reveals whether semantic similarity is overwhelming exact identity.
Choose rollout criteria that match the task
Set thresholds for evidence recall, correct-entity retrieval, freshness and latency before release. Include no-answer questions and users with different access rights. A blended relevance score alone cannot establish those properties.
If one route fails, record which route was unavailable. Decide whether partial results are acceptable for the task, and avoid presenting a degraded search as a complete corpus review.
For context boundaries, see LuMay's RAG chunking guide. Start your hybrid-search pilot with the exact identifiers and paraphrases that currently fail; they provide a more useful benchmark than a collection of easy demo questions.
