Gains:
- Designing the components and data flow of an end-to-end enterprise RAG assistant
- Combining multi-source data (wiki, ticket, PDF, database) into a single assistant
- Make architectural decisions for scalability, caching and latency
In previous units, we learned the parts one by one: embedding, vector database, chunking, retrieval. Now let's combine these and build an end-to-end architecture of an assistant that talks to your own company data. The goal is to have an employee ask, “What is our leave policy?” A system where people can ask questions, the answers are based on real internal documents, citations and combine multiple data sources. This unit processes the entire architecture, data flow and production level decisions.
End-to-End Components
A corporate RAG assistant consists of two separate lines. The indexing line (offline) prepares the data; The query line (online) answers the question.
Indexing line components:
- Connectors: Connectors that pull data from sources — wiki, ticket system, file store, database, email.
- Normalization: Converting different formats (PDF, HTML, DOCX) to clean text; header/footer cleaning.
- Chunking + metadata: Chunking and tagging (source, date, authority).
- Embedding + loading: Writing vectors and metadata into the vector database.
Query pipeline components:
- Query preprocessing: Rewriting, decentralization.
- Retrieval: Hybrid search + metadata filter + re-ranking.
- Prompt creation: Placing context + question + instructions into the template.
- Generation: Grounded (contextual) answer from the model + sources.
- Post-processing: Citation formatting, security check, logging.
Tip: Physically separate the indexing line from the query line. Indexing is slow and periodic (runs in batches overnight); The line of inquiry should be light and immediate. Mixing the two lines forces heavy processing while the user waits.
Visualizing Data Flow
[INDEXING - offline]Resources → Normalize → Chunk+Metadata → Embed → Vector DB (wiki, ticket, PDF, DB)[QUERY - online]User question → Pre-processing → Retrieval (hybrid+filter+rerank) → Prompt (context+question+instruction) → Model → Answer+Source → User
Combining Multi-Source Data
In real companies, the answer doesn't stop in one place. “How to issue a refund to a customer?” The answer to the question can be found both in the help article (procedure), in the ticket history (real examples), and in the policy PDF (rules). The assistant should search all of them in one pool.
Critical point: when combining resources into a single vector store, each shard must carry the `source_tour` metadata. So you can search them all and filter them if necessary, such as "only bring official policies". Also, different sources have different levels of reliability: official policy > help article > an employee's ticket note. You can specify this priority in re-ranking or prompt.
Source
Content type
trust
Update frequency
Policy PDF
official rule
high
monthly
Help article
Procedure
medium-high
weekly
Ticket history
real sample
medium
Continuous
wiki
Mixed/current note
Variable
Continuous
Scalability, Cache and Latency
Three issues stand out in production. Latency: The experience deteriorates when the user waits more than 2 seconds. Solution: display the answer in streaming form — it is poured onto the screen as the model writes. Cache: For frequently asked questions and repetitive contexts, cache both increases speed and reduces cost. Scale: As the user increases, it is necessary to be able to scale retrieval and model calls horizontally.
Rule of thumb on the cost side: the most expensive step is usually the number of tokens going to the larger model. Therefore, reducing the context to 4 good parts by re-ranking improves both quality and cost. A common design is to use a smaller/faster model for simple classification or routing, and a more powerful model for the final answer (e.g. claude-opus-4-8).
Caution: Do not set up indexing as "do it once, forget it". Documents are changed, deleted, added. Establish a re-indexing strategy: detect changed documents and reprocess only them. The stale index produces an answer that appears current but is wrong.
Weak Architecture / Strong Architecture
Weak (single script, everything mixed):
When the user asks: read the documents at that moment, shred them, embed them, search them, answer them.# Problem: all indexing is repeated for each question; seconds of delay, # no source separation, no filter, no refresh.
Powerful (split pipes + metadata + cache + streaming):
Indexing: batch runs at night, refreshing changed documents. Query: lightweight line — pre-processing → hybrid retrieval+filter → rerank → prompt → model (streaming) → citation → log. Frequently asked questions and source are cached.
Three Mini Cases
Case 1 — Confused line, heavy delay. A startup wrote a script that reprocesses PDFs with each question; Each answer took an average of 11 seconds. When the indexing line was separated and the data was previously transferred to the vector store, the query time decreased to 1.3 seconds and with streaming, the "first word" appeared in 400 ms.
Case 2 — Too many resources, wrong priority. A support assistant gave equal weight to the policy PDF and old ticket notes; The model sometimes presented an employee's incorrect rating from two years ago as the official rule. When source_tour metadata and the "consider official policy in case of conflict" instruction were added to the prompt, false-priority errors were reduced by 89%.
Case 3 — Stale index. An HR assistant was working with an index that was not updated for 3 months; The leave policy has changed, but the assistant was saying the old days. When daily refresh was installed, which detects changed files, the current-response rate increased from 70% to 99%.
Common mistakes
- Mixing indexing and query lines: Heavy processing is done while the user waits; delay explodes.
- Not putting the source type in metadata: No prioritization and filtering; The untrusted source appears to be official.
- Not establishing a refresh strategy: The index becomes stale; Wrong answers that appear current are produced.
- Skip streaming: User looks at a blank screen; The perceived delay becomes high.
- Using the largest model at each step: Cost increases unnecessarily; Leave the steering to the smaller model.
In summary
- The corporate RAG assistant consists of two separate lines: offline indexing and online query; separate them physically.
- Indexing = connector + normalize + chunk/metadata + embed/upload; query = pre-process + retrieval + prompt + generate + post-process.
- Multi-source data is combined into a single repository, but source_type metadata and trust priority are preserved.
- Streaming and cache for latency, context throttling and model selection for cost are critical.
- Without re-indexing, the index becomes stale; Reprocess changing documents regularly.
Application task
Draw an architectural diagram of an assistant for your own team. (1) Identify at least three real data sources and write down a connector need, update frequency, and trust level for each. (2) Draw the indexing and query lines separately with a box-arrow diagram. (3) “Where do I reduce latency and cost in this assistant?” Write at least two concrete decisions to the question. (4) Describe your refresh strategy in one sentence: which resource will be reindexed and how often?
checklist
- [ ] I can draw the indexing and query lines separately and with the correct components.
- [ ] I can combine multi-source data with source_type and trust priority.
- [ ] I can make streaming/cache decisions for latency and model selection for cost.
- [ ] I know why a re-indexing strategy is essential.
- [ ] I keep in mind that the most expensive step in my architecture is usually the token that goes to the larger model.