A RAG pipeline is the workflow that retrieves relevant information from approved external sources and supplies that evidence to a large language model before it answers. For enterprises, RAG makes AI more useful for current, private, and domain-specific knowledge while improving source traceability without requiring the model to memorize every document.
The pattern comes from retrieval-augmented generation research. Lewis et al.'s original RAG paper combined language-model memory with retrieved external knowledge, while Microsoft Learn describes modern RAG as grounding model responses in retrieved content.
For this guide, stages 1–4 are indexing and stages 5–7 are query-time processing. This is a practical teaching model, not a universal standard.
RAG Pipeline at a Glance
| Stage | Input | Output |
|---|---|---|
| Stage 1: Ingest and clean | Documents, databases, APIs | Normalized source content |
| Stage 2: Chunk and add metadata | Clean documents | Searchable passages with source and access metadata |
| Stage 3: Generate embeddings | Text chunks | Vector representations |
| Stage 4: Build the search index | Chunks, vectors, metadata | Searchable, filterable index |
| Stage 5: Process question and retrieve | Question plus user identity | Authorized candidate passages |
| Stage 6: Rerank and assemble context | Retrieved candidates | Ordered evidence package |
| Stage 7: Generate grounded answer | Question plus evidence | Answer with sources, or an abstention |
A Running RAG Example
Imagine an employee asks: “Can I take my company laptop on an international business trip?”
A fictional current policy, IT-TRAVEL-004 v3, says company laptops may leave the country only when manager approval is recorded in the IT portal at least five business days before travel and full-disk encryption is active.
The knowledge base also contains an older v2 policy and a restricted security document. A well-designed RAG pipeline should retrieve the current policy, exclude the superseded version, and prevent restricted content from reaching an unauthorized user's model context.
The 7 Stages of a RAG Pipeline
Stage 1: Ingest and Clean Source Data
RAG starts by connecting to approved sources such as PDFs, wikis, SharePoint libraries, databases, cloud storage, APIs, manuals, and enterprise applications.
The ingestion layer extracts usable content, removes duplicate or obsolete material, normalizes formatting, and records source identity, ownership, revision date, and classification. Scanned or layout-heavy documents may require OCR or layout-aware extraction.
LuMay's RAG development services include extraction, cleaning, normalization, deduplication, metadata enrichment, OCR, and document processing.
Stage 2: Chunk Documents and Attach Metadata
Large documents are divided into smaller passages so retrieval can return the section that actually answers a question. Good chunking preserves enough context for meaning without making each passage unnecessarily broad.
Microsoft's RAG chunking guidance recommends testing chunking strategies against real documents and queries.
For IT-TRAVEL-004 v3, the international-travel rule becomes a chunk tagged with source ID, section, version, effective date, owner, and access group. That metadata later supports filtering, freshness, permissions, and citations.
Stage 3: Generate Embeddings
An embedding model converts text into numerical vectors that represent semantic relationships. This helps retrieval find conceptually similar passages even when the user's wording differs from the source.
Document chunks can be embedded during indexing, and the user's question can be converted into a query embedding at runtime.
Embeddings are common, but not mandatory. Exact identifiers, names, and policy numbers may benefit from keyword search, and many enterprise systems combine keyword and vector signals.
Stage 4: Build the Search Index
The system stores chunks, metadata, and optional vectors in a searchable index.
A vector database is one implementation choice, not the definition of RAG. Retrieval can use keyword, vector, semantic, graph, relational, or hybrid search. Microsoft documents vector, keyword, and hybrid retrieval as options to test and tune for the workload.
Production indexes should make fields such as source, version, department, date, and access level filterable. LuMay's AI integration services connect AI systems with enterprise data, applications, APIs, and workflows.
Stage 5: Process the Question and Retrieve Authorized Content
At query time, the application interprets the question, may rewrite or expand it, creates a query embedding where appropriate, and searches the index.
Security must happen during retrieval. Permission filters should be applied before content reaches the LLM. Microsoft documents document-level access control for excluding content a caller is not authorized to see.
In our example, access filters remove the restricted security document, while version filters remove IT-TRAVEL-004 v2. The current v3 rule remains.
Stage 6: Rerank Results and Assemble Prompt Context
Initial retrieval often favors recall. A reranker can then reassess candidate passages and move the strongest evidence to the top.
Reranking is optional because it can improve precision but also add latency and cost. The application then assembles a compact context containing the question, selected evidence, source identifiers, and instructions for using that evidence.
Retrieved text should be treated as untrusted data. OWASP's prompt-injection guidance notes that RAG does not fully eliminate prompt-injection risk.
Stage 7: Generate an Evidence-Grounded Answer With Sources
The LLM receives the question plus selected evidence and generates an answer grounded in that context.
For the fictional example:
Yes, with two conditions. Your manager's approval must be recorded in the IT portal at least five business days before travel, and full-disk encryption must be active. Source: IT-TRAVEL-004 v3.
If the evidence does not answer the question, the system should abstain rather than invent a response. RAG reduces unsupported answers, but it cannot guarantee factual accuracy.
RAG vs. Search vs. Fine-Tuning
| Capability | Ordinary Search | RAG | Fine-Tuning |
|---|---|---|---|
| Primary goal | Find documents | Answer from retrieved evidence | Change model behavior |
| Output | Links or ranked results | Natural-language answer with sources | Responses shaped by additional training |
| Knowledge updates | Update search corpus | Update knowledge base and index | Retrain or further train |
| Best fit | Finding information | Current, private, verifiable knowledge | Style, format, specialized behavior |
RAG builds on retrieval rather than replacing search. It can also complement fine-tuning when a system needs both current knowledge and specialized model behavior.
Common RAG Failures and Production Checks
| Failure | Production Check |
|---|---|
| Stale content | Version filters, refresh jobs, retired-source removal |
| Poor retrieval | Test chunking, hybrid search, filters, query rewriting, reranking |
| Permission leaks | Apply identity-aware filters inside retrieval |
| Prompt injection | Isolate retrieved content, limit tools, validate outputs |
| Unsupported answers | Use groundedness checks and abstention rules |
| Citation errors | Validate every source against retrieved context |
How to Evaluate and Monitor a RAG Pipeline
Measure the pipeline stage by stage rather than judging only whether the final answer sounds good.
Track:
- Retrieval relevance: Did the system retrieve the right passages?
- Groundedness: Is the answer supported by the retrieved evidence?
- Citation accuracy: Do references substantiate the associated claims?
- Permissions: Can users retrieve only authorized content?
- Latency: How long do embedding, search, reranking, and generation take?
- Cost: What does each query cost across the pipeline?
Microsoft Foundry includes RAG evaluators for retrieval, groundedness, and relevance. LuMay's RAG offering also covers retrieval relevance, groundedness, citation accuracy, latency, cost, index refreshes, monitoring, and optimization.
Build a Production RAG Pipeline With LuMay
LuMay's RAG development services cover enterprise knowledge integration, document preparation, embeddings and vector implementations, custom retrieval, LLM integration, evaluation, deployment, and ongoing optimization.
These capabilities are supported by broader AI engineering and integration services for connecting models to enterprise systems and workflows.
If you are planning an enterprise knowledge assistant, internal search experience, or RAG-enabled application, book a LuMay consultation to review data readiness, retrieval architecture, permissions, evaluation, and production deployment.
