Gains:
- Understanding the logic of knowledge-based generation (RAG) and why it reduces hallucinations
- Ability to have the model respond with citation, based only on the source documents provided
- Ability to manage questions that are not in the source by transferring them safely, without making up questions
A language model "knows" a lot, but it doesn't know your company's return policy today, your current price list, or your shipping agreement that changed yesterday. Worse, when he doesn't know, he often makes it up and writes in an extremely confident tone. This is a fatal flaw in customer service: the model says "Refunds are issued within 30 days" while your policy may be 14 days. The approach that closes this gap is called RAG: Retrieval-Augmented Generation, that is, "retrieval-augmented production".
The idea is simple: The model answers the question not from memory, but from the correct source documents placed in front of it at that moment. First, the correct piece of document relevant to the question is "retrieval", then the model "generates" the answer based on that piece only. In this unit, you will understand the logic of RAG and learn how to rein in the model so that it only sticks to the verified source you provide.
Note: This unit teaches the working logic and prompt side of RAG. Enterprise-scale automatic document retrieval systems (vector database, etc.) require a technical installation; Discipline here is the basis for that system to respond correctly.
Why Does RAG Reduce Hallucinations?
Hallucination is when the model produces unreal information as if it were real. The model is programmed to produce a response that seems “possible” when it sees a gap; Doesn't know if it's true or not. RAG fills this gap: you give the model the actual relevant text along with the question and say “only answer from here.” This way the model doesn't "need" to fit.
There are three components:
- Knowledge base: Reliable texts such as FAQs, policy documents, product manuals, price lists.
- Retrieval: Finding the piece of document containing the answer to the question and giving it to the model.
- Limited edition: Model responds with quote, based only on the part supplied.
Step by Step: A Faithful Response to the Source
- Prepare the source. Find the text on which the response will be based (policy, FAQ) in a clear and up-to-date format.
- Embed the source in the prompt. Put the text in a tag like <source>.
- Write the loyalty rule. "Only use the information from the source; do not stray from the source."
- Make citation required. Let him/her indicate which section the answer is based on.
- Identify non-resource behavior. If the answer is not in the source, let him say "I don't have this information" and pass it on.
- Verify. Confirm the answer by comparing it with the source sentence on which it is based.
Copiable Prompts
Response prompt based on the underlying source:
Role: You are a customer support assistant. Answer the question ONLY based on the information in the <source>block below. Do not add any information that is not in the source, guess or answer from your general knowledge. Add the section you are relying on at the end of your answer in the format "[Source: ...]". If the answer is not in the source, write the following: "I do not have definitive information on this subject, I am transferring you to an official who can give the correct answer."<source>{{ policy_or_faq_text }}</source>Question: {{ customer_question }}
In case of multi-source, the prompt to show which document it is based on:
There are multiple resources numbered below. When answering the question, state the number of the source you used, such as [Source 2]. If more than one source contradicts you, say so clearly and state that you should ask which one is more up-to-date.<sources>1) {{ source_1 }}2) {{ source_2 }}3) {{ source_3 }}</sources>Question: {{ customer_question }}
Prompt that safely manages the non-source question:
If the answer to the question is PARTIALLY found in the source: answer only the part covered by the source, and say "this detail is not in the source" for the remaining part. Never fill in the missing part by guessing.
Prompt that translates the response into the client language, but stays true to the source:
Translate the official/technical expression in the source into plain language that the customer can understand, but DO NOT change the meaning and numerical values (day, amount, rate). For example, rounding "14 calendar days" to "about two weeks"; keep full value.
Weak Prompt / Strong Prompt
Weak prompt
Powerful prompt
"Tell us about our return policy"
Embeds source + "reply here only" + requests quote
Answer from memory of the model (maybe wrong)
Answer from your current document
If it's not in the source, he makes it up
He says "I don't have this information" and passes it on.
Can round and distort numbers
Maintains day/amount/rate exactly
The difference is that the strong prompt gives the model an anchor (source text) and a ban (going off source). The model no longer speaks from memory, but from the reality in front of him.
Three Mini Cases
Case 1 — The old policy trap. "Returns accepted within 30 days," one store's bot said while running without credit; However, the company reduced the period to 14 days. When the updated policy text is added to the source in the RAG installation, the bot now responds "14 calendar days [Source: Return Policy, article 2]". Refund disputes arising from false promises have been reset.
Case 2 — Catching the contradiction. A customer asked about shipping costs. Both the old and new price lists were entered into the system as sources. Thanks to the multi-source prompt, the model said, "Two sources give different prices (49 TL and 59 TL); I need to confirm the current one" and conveyed the issue to the human. Instead of telling the customer an incorrect amount, he communicated the uncertainty honestly.
Case 3 — Partial response discipline. “Do you ship the product abroad and who pays the customs duty?” In the question, the source only said that the shipment was made, there was nothing about who paid the tax. The model said, "Yes, shipping is done abroad [Source: Cargo FAQ]. However, it is not stated in this document who pays the customs duty; I am transferring you to the official to clarify this." Instead of a half-true, half-fabricated answer, it produced an honest and confident output.
Tip: Keep the most current and single source of truth version of your knowledge base. Having the same information (e.g. return period) written differently in three separate documents is the biggest enemy of RAG; The model responds differently depending on which one it looks at. First deduplicate your documents, then automate them.
Verification: Don't Relax Just Because the Source Exists
RAG greatly reduces the hallucination but does not reset it. The model may sometimes misinterpret the source, mixing up two sentences, or drawing a “conclusion” that is not in the source. That's why the citation requirement is critical: open the section on which the answer claims to be based and check whether it actually says so. This control is non-negotiable, especially in responses containing numbers (day, amount, rate) and conditions (exception, condition).
Caution: If the source itself is wrong or outdated, RAG faithfully repeats that mistake. Saying "the model speaks from the source" does not mean "the model speaks correctly"; You are responsible for the currency and accuracy of your source.
Keeping the Knowledge Base Ready for Response
The quality of RAG depends largely on how the source text is written. The model returns a much more accurate response than well-structured, titled, single-topic texts. Practical advice: write your policies in short, titled sections rather than long, nested paragraphs; Let each section answer a single question ("How many days is the return period?", "Which products cannot be returned?"). Keeping frequently asked questions (FAQs) in a question-and-answer format makes it easier for the model to find the right part. State values such as date, amount, and rate clearly and in one place within the text; Writing the same number differently in different sections confuses the model.
Another important habit is to keep the knowledge base alive. Policies, prices and promotions vary; If the source is not updated, the model continues to safely repeat the old truth. With each policy change, update the source and re-ask the model a few test questions about the changed topic to confirm that it gives the correct answer. This small but regular maintenance prevents the RAG system from "silently going wrong" over time.
Common mistakes
- Saying "explain our policy" without burying the source and relying on the model's memorization.
- Not defining the "reply from source only" boundary and non-source behavior.
- Not requesting a quote/source reference and leaving the answer unverifiable.
- Keeping conflicting versions of the same information in multiple documents in the knowledge base.
- Allowing the model to round/interpret numerical values.
- Forgetting to update the source, causing the model to faithfully repeat old information.
In summary
- RAG means that the model produces its answer from the current source document, rather than from memory.
- The most practical way to reduce hallucination: bury the source, say "only reply here", ask for citation.
- If the answer is not in the source, one should not fit a model; He should say "I don't have it" and hand it over.
- Make the model aware of conflicting sources; You preserved the numerical values exactly.
- RAG accuracy depends on the timeliness of your source; Always confirm the quote.
Application task
Take an actual policy text (return, shipping, or membership) from your own business and embed it as a source in the basic RAG prompt above. Then ask three questions: (1) a question whose answer is clear in the source, (2) a question whose answer is not at all in the source, (3) a question whose answer is only incomplete in the source. Verify that the model answers correctly, says “I don't have it,” and honestly marks the partial answer and the missing, respectively. Correct deviant behavior by strengthening the loyalty and non-resource rules in the prompt.
checklist
- [ ] I embedded the current source on which the response will be based in the prompt.
- [ ] I added the "Only reply from the source, don't go beyond" rule.
- [ ] I have put a source reference / citation requirement.
- [ ] I defined delegation behavior if the answer is not available in the source.
- [ ] I required that numerical values be preserved exactly.
- [ ] I confirmed the answer from the source sentence on which it is based.