Gains:
- Explain the causes of hallucination and the role of grounding
- Applying techniques that make the model say 'I don't know' if it is not in context
- Ensuring trust and auditability by adding verifiable sources/citations to responses
The most dangerous behavior of a language model is to make up something he does not know in a confident and persuasive tone. This is called hallucination. Hallucinating in a corporate assistant is not just annoying, it's downright risky: the wrong day off, the wrong price, the wrong legal substance can actually put an employee or customer in harm's way. In this unit, we cover why hallucinations occur, how to reduce them in RAG, and how to make the answer verifiable.
What Causes Hallucinations?
The model is trained to produce the next most likely word; Not "tell the truth", but "produce fluent and reasonable text". If information is missing, it can fill the gap with something statistically plausible but factually incorrect. Three typical triggers:
- There is no answer in the context, but the model feels compelled to say something anyway.
- The context is conflicting or inadequate; The model closes the gap with prediction.
- The question goes beyond context; the model returns to its own general (and possibly outdated) knowledge.
RAG reduces hallucination because it gives the model a real source — but it does not reset it. If you bring the wrong part or leave the model "freedom to fill the gap", the system with RAG can also fit it.
type of hallucination
symptom
root cause
solution layer
originating from retrieval
The correct answer is in the document but it did not arrive
Wrong/missing part delivered
Chunking, hybrid, re-ranking (Unit 3-4)
Generation sourced
The correct part arrived but the answer is wrong/must be added
The model filled the gap with prediction
Grounding, "I don't know" permission, low creativity
Citation sourced
The answer is correct but the source is wrong
Model referred to wrong part
Citation verification (see Unit 8)
Hint: Divide the hallucination into two parts. Sourced from retrieval: wrong/missing part arrived (solution: improve retrieval). Generation source: The correct part arrived, but the model read/added it incorrectly (solution: prompt and grounding). You can't fix it unless you know which one it is.
Grounding: Nailing the Answer in Context
Grounding is the name given to all techniques of telling the model "just rely on the text I give you, don't go beyond it." His most basic and most effective technique: explicitly giving the model permission to say 'I don't know'. The model, like the human, is hesitant to say "I don't know"; You must give him express permission to do this.
Answer only based on the CONTEXT below. If the answer is not CLEARLY in the context, write exactly this: "I couldn't find enough information on this in the documentation." Do not include any numbers, dates, or names that are not in the context. Don't guess.
Additional grounding techniques:
- Quotation requirement: Before each claim, quote the context verbatim (“The document says: '...'”). Producing fabricated quotes is more difficult for the model.
- Low temperature: The temperature (temperature, creativity/random setting) should be low. Note: in some current models this parameter has been removed; In them, you provide grounding with a prompt. High creativity is the enemy of corporate integrity.
- Scope limit: "Only respond to consent policy issues; politely decline to opt out."
Citation
The strongest systemic defense against hallucination is to make the answer verifiable. If the user can see which document the answer is based on: (1) he can catch the mistake himself, (2) audit becomes possible, (3) trust increases, and (4) the model produces more cautiously with the awareness that "I will need to cite the source".
Two common approaches:
Inline attribution: Source tag next to each claim. "Annual leave is 14 days [1]." Below is the full documentation of [1].
Structured output: Asking the model to produce the answer and the source list in separate fields; Showing resources as clickable links in the interface.
# Producing a structured answer based on the model (conceptual) Give your answer in the following structure:- answer: <context-based answer>- sources: [{"part_no": 1, "file": "...", "section": "..."}]- trust: <high|medium|low> # How clearly does the context support it? If the context does not support the answer, write "no information found" in the answer field and leave the sources blank.
For Citation to really work, it is necessary to verify that the attribution is correct. The model may sometimes give the correct answer but give the wrong source. Advanced systems automatically check that each claim produced is actually supported in the piece it points to (the "faithfulness" measurement in the next unit).
Weak/Strong: Prompt Against Hallucination
Weak (invitation to fill in the blank):
Answer the question as best you can with the information you have: {context} Question: {question}# "As best you can" means "guess if you don't know" for the model.
Strong (grounding + don't know permission + source + trust):
Just rely on CONTEXT. Otherwise, say "I couldn't find any information"; make up number/date/name.Add [n] source number to each claim. Specify the level of trust supported by the context.CONTEXT: [1]... [2]... QUESTION: {question}
Three Mini Cases
Case 1 — Made-up item number. A paralegal made up an "Article 17/B" statement that was out of context and gave false information to the employee. When the instruction "Don't give an article number that is not in the context, otherwise say I don't know" + the obligation to quote was added, the cases of fabricated articles decreased from 11 to 0 in 40 examples.
Case 2 — Correct answer, wrong source. A support assistant stated the correct return period but cited the wrong article; When the customer clicked on the link, an irrelevant page opened. When citation verification (checking whether the claim appears in the cited piece) was added, the false-citation rate dropped from 23% to 2%.
Case 3 — “I don't know” permission was not granted. An HR assistant had made up a plausible but incorrect answer to a question that was not in the document. When explicit "I don't know" consent and full text ("I couldn't find information about this in the documentation") were added to the prompt, the honest rejection rate for questions with no answer, rather than fabrication, increased from 8% to 95%.
Common mistakes
- Not allowing "I don't know": The model fills the gap with fabrication.
- Keeping creativity high: Randomness is detrimental to corporate integrity.
- Trusting the source without verifying it: The model may attach the wrong source to the correct answer.
- Not distinguishing between the two types of hallucinations: You correct the wrong place without knowing whether it is caused by retrieval or generation.
- Mistaking Citation for cosmetics: Citation of source is both confirmation and deterrent; take it seriously.
Caution: "The model looks very confident" is not the same as "the model is correct". Hallucinations are generally the most fluent and confident sentences. Read the trust from the source it indicates, not from the tone of the sentence.
In summary
- Hallucination is when the model fills the information gap with plausible but false text; RAG reduces but does not reset.
- The hallucination may be caused by retrieval (wrong part) or generation (wrong reading); Determine which one first.
- Grounding's most powerful technique is to give the model explicit "I don't know" permission; also citation requirement and low creativity.
- Citation makes the answer verifiable; It both gives confidence and pushes the model to be produced cautiously.
- The accuracy of the citation should also be checked; The wrong source may be attached to the correct answer.
Application task
(1) Prepare 4 questions to ask your assistant, the answers to which are not in the documents (trap questions). Test each one with the weak and strong prompt and mark whether the model fits or not. (2) For the 4 questions whose answers are in the document, have the model generate the weld number and manually verify whether the weld it shows is actually the correct part. (3) Check whether the model gives "right answer, wrong source" in at least one case and write in one sentence how you can catch this automatically.
checklist
- [ ] I can tell why the hallucination is happening and that RAG reduces it but does not reset it.
- [ ] I can distinguish between retrieval and generation-induced hallucinations and correct the correct place.
- [ ] I am adding explicit "I don't know" permission and grounding rules to the prompt.
- [ ] I am adding a verifiable source (citation) to the answer.
- [ ] I understand the need to further check the accuracy of the attribution.