Gains:
- Ability to explain why hallucination (fabricated source, quote, event) and anachronism are the most dangerous mistakes in history
- Ability to apply the discipline of verifying each reference in the catalog, comparing content to the document, and cross-confirming critical facts.
- Ability to question bias and missing voices in sources by asking artificial intelligence where to look instead of the real source
There is a caveat repeated in every unit of this module: the output of the AI is a claim, not a fact, until verified. This unit turns that warning into a discipline. History and archiving are inherently evidence-based fields; The most basic skill of a historian is source criticism (questioning by whom, when, for what purpose a source was produced and how reliable it is). In the age of AI, this skill has become even more critical because the source itself may now be fabricated.
Here we will cover three dangers in depth: hallucination (AI producing fabricated information, sources or quotes), anachronism (periodical elements being moved to the wrong period), and the discipline of source verification in general. We will also see AI learning from sources that were inaccurate or biased in the past, so it can reproduce bias.
Hallucination: the most dangerous mistake
A hallucination is when an AI confidently presents something that does not actually exist. Historically, it is most commonly seen in the following forms:
- Fabricated source/reference: A non-existent archive fund, file number, book or article.
- Made-up quote: A word that was not actually said, a sentence that is not in the document.
- Made-up event/date/person: An event that did not happen, a false date, a person who does not exist.
- Fictitious connection: A non-existent cause-effect relationship between two real events.
What makes a hallucination dangerous is that it seems believable. A made-up reference has the same format as a real reference: fund name, number, date. So "format looks right" never means "correct".
Attention: No resource produced by YZ can be used without being verified and found in the catalog or in a reliable place. The only way to prove that a reference exists is to see it where the original source is located (archive catalogue, library, publication). "AI gave it" is not evidence.
Anachronism: glasses of the period
Anachronism is seeing the past through today's glasses. Because AI is trained in today's language, it has a strong tendency to:
- Term anachronism: Using a word, institution or concept that does not belong to the period.
- Concept anachronism: Projecting an idea of the present (e.g. a modern ideology or identity) into the past.
- Value anachronism: Judging past behavior only by today's standards.
The historian's job is to understand the past in its context. AI lacks this context sensitivity; You must provide it yourself.
Understanding why hallucination happens so often makes it easier to protect against it. AI is a system that works to produce the “next most likely word”; It has no purpose like "knowing the truth". When asked a question, whether he knows the answer or not, he produces text that looks statistically convincing. The format of an archival reference (fund name, number, date) is a learned pattern; AI can fill this pattern without a real reference. So “why does AI make it up?” The answer to the question is simple: it is easier for him to make up than "not knowing". The responsibility falls on you to fill these molds with real resources.
Source verification discipline: step by step
1. Verify the existence of the resource. Find every reference given by AI in the catalog/publication. If you can't find it, don't use it.
2. Compare the content with the source. Does the information that AI attributes to a document actually exist in the document? Is the quote exactly the same?
3. Cross-confirm. For a critical fact, look at at least two independent reliable sources.
4. Apply source criticism. Is the source itself reliable? Who produced it, when and for what purpose? Is it biased?
5. Pass it through the anachronism filter. Are there any terms, concepts or value judgments that do not fit the period in the output?
6. Question prejudice. Could AI have reproduced the biases in historical sources (ignoring one group, favoring one point of view)?
Authentication layer
Question
who does
entity
Does this resource really exist?
You in the catalogue.
Content
Does this really say in the document?
You in the document
Confirmation
Does any other source support it?
you, cross
reliability
Is the source reliable?
expert judgment
Period
Are there any anachronisms?
expert judgment
prejudice
Whose voice is missing/distorted?
expert judgment
three mini cases
Case 1 — Six fabricated references. A researcher asked the AI for a list of sources on a topic. YZ provided 6 archive references with perfect formatting. In the catalog search, 4 of the 6 were missing, 2 of them had the wrong number. If the researcher had not verified, a study that cited non-existent sources would have emerged. Lesson: every reference is verified in the catalogue.
Case 2 — Made-up quote. A writer would use a striking quote that the AI attributes to a historical figure. When he investigated the source, he saw that there was no document proving that person said such a thing, and that the statement was "made up" by AI. Lesson: quotes cannot be used without verification from a primary or reliable source.
Case 3 — Reproduction of prejudice. A student had the AI explain the history of a region. The output completely ignored a specific community in that area because the sources the AI had learned from had also ignored them. The student noticed and corrected this gap with cross-references. Lesson: AI reproduces silences and biases in sources; One must ask whose voice is missing.
Four copyable templates
1) Reference verification list:
ALL source, reference, quote, date and person names in the text below appear in a separate list. Add a note for each: "claimed in this text, must be independently verified." If you select none, "verify"; Just produce a list to verify so I can check it from the catalog/source. Text: [here]
2) Anachronism scanning:
Examine the text about [period] below. Highlight terms, institutions, concepts, and values that MAY NOT belong to this period. For each, write briefly why you are suspicious. Making final judgment; Create a list of suspicions that the expert will check. Text: [here]
3) Source criticism framework:
A source critique is drafted about the following source: (1) Who might have produced it, (2) When, (3) For what purpose, (4) What are its possible biases, (5) Which issues should be trusted and which should be cautious. Imposing final judgment; Provide questions and possible answer options. Source: [here]
4) Missing volume/bias control:
WHOSE voice and which group's experience might be missing or one-sided in the following historical summary? Which perspective is dominant, which is invisible? Produce this as a list of questions; I will fill it with cross-references. Summary: [here]
Weak prompt / Strong prompt
Weak:
Give me reliable historical sources and quotes on this subject.
AI is very prone to produce references that are correct in form but fabricated in content when asked for "source" and "citation". This prompt is a direct invitation to hallucination.
Strong:
Suggest what KIND of sources (archive fund type, publication type, institution) I should look for in my research on this subject. FITTING a specific reference, file number or quote; Just tell me where to look. Then I will find the real sources from the catalogs.
Difference: AI is asked for "where to look", not "actual reference"; The risk of hallucination is radically reduced and verification is left to the human.
Common mistakes
- Relying on the correct reference. The made-up reference also appears flawless; must be verified in the catalogue.
- Taking a direct quote from AI. Quotations cannot be used without confirmation from the primary/reliable source.
- Overlooking the anachronism. Today's terms and concepts silently seep into the past.
- Reproducing prejudice. AI copies silences from sources; It is necessary to ask about the missing sound.
- Relying on a single source. Critical facts must be confirmed by at least two independent sources.
In summary
Source verification is at the heart of history and archiving, and is even more critical in the age of AI. Hallucination produces sources, quotes, and events whose form is perfect but whose content is fictitious; anachronism distorts the past through today's lenses; and AI reproduces biases in sources. Defense is a discipline: verify each reference in the catalog, compare content to document, cross-check critical facts, filter through period and bias. Ask the AI for “where to look,” not “the actual source.” The guardian of truth is always the expert.
Application task
Have the AI produce a list of resources on a topic you are working on (with a deliberately "weak prompt"). Then extract all the references from this output with the "reference validation list" template and search each one in a catalog. Count how many actually exist and how many are made up or inaccurate. Repeat the same topic with a "strong prompt" ("where should I look") and experience the difference.
checklist
- [ ] I verified each reference in the catalog/trusted source.
- [ ] I have verified each quote with the primary/reliable source.
- [ ] I have cross-checked critical facts with at least two independent sources.
- [ ] I filtered the output for anachronisms.
- [ ] I questioned missing/distorted voices and bias.