Gains:
- Ability to extract patterns from large sets of text using techniques such as NER, relationship extraction, timeline and network analysis
- Being able to see the pattern as a hypothesis rather than evidence, linking entities to the source and normalizing them with human approval
- Ability to understand how numbers can be misleading due to missing data and sampling bias, and to avoid contrived relationships and chronologies.
Historical research is not just about reading individual documents; often looking for patterns in large sets of documents. In what contexts does a particular name appear over the years? Which people are associated with a settlement? How does a topic change over time? Who is corresponding with whom? Such questions require looking at not a single document, but hundreds of documents collectively. This is often called digital humanities—the application of computational methods to fields such as history and literature.
AI is powerful at extracting structured information from large sets of text. In this unit, you will learn NER (named entity recognition), pattern and relationship extraction, topic modeling logic and timeline creation; but we will also discuss the most dangerous pitfall of these techniques—the AI presenting itself as having “found” a pattern that does not exist.
Basic techniques
- NER (Named Entity Recognition): Automatic extraction of entities such as person, place, institution, date from the text. It produces a list like "All place names mentioned in these 500 documents" in minutes.
- Relationship inference: Marking ties between entities (“Person X at place Y”, “A corresponds with B”).
- Topic modeling: Statistically finding clusters of recurring topics in a large set of text. It shows which themes are concentrated in which period.
- Timeline inference: Arranging dated events in documents into a chronological order.
- Network analysis: Visualizing inter-person/institutional relationships as a network (who is the center, who is the connection point).
The common goal of all of these is to reveal structures that do not appear in a single document but appear throughout the collection. These techniques make visible patterns that would never be visible through reading alone. But each of them is a "sign", not a "evidence"; It is essential to verify what is found in the original document.
A frequently used concept in this field is the distinction between close reading (reading a text in depth, line by line) and distant reading (looking at hundreds/thousands of texts collectively, statistically). AI makes it possible to read remotely: you see patterns in a corpus large enough to last a single human lifetime. But when distant reading points to a pattern, it is necessary to return to close reading, that is, to the original document, to make sense of it. The two methods are not interchangeable; One shows the pattern, the other gives its meaning. The most robust research uses both together.
Tip: Use pattern analysis as a hypothesis generator: “The AI has spotted a pattern here, is it real?” Then check that pattern in the documents one by one. The pattern is the starting point of your research, not the result.
Workflow: step by step
1. Clarify the question. “Which people appear together most frequently in this collection?” A clear pattern question like.
2. Prepare and label the data. Organize text with source references so that any assets found can be linked back to the document.
3. The entity/pattern appears. Generate NER, relationship or timeline outline with AI.
4. Normalize. Combine different spellings of the same person/place (“Istanbul / Istanbul / Kostantiniyye” the same place?). This step is critical; If it is omitted, the pattern will fall apart.
5. Verify in document. Check the actual documents for any significant patterns that are flagged. Make sure the AI doesn't add a made-up link.
6. Comment. What the pattern means is the historian's judgment; AI just marks the pattern.
Number and statistics trap
Pattern analysis often produces numbers: “name X occurs 47 times”, “this theme is present in 30% of documents”. These numbers seem objective but can be misleading:
- Missing data: If the collection is already incomplete (documents have been lost, a group has never been recorded), the number reflects surviving documents, not reality.
- Normalization error: If the same person is spelled differently and counted separately, the number will be incorrect.
- Loss of context: "occurs 47 times" doesn't say anything; How it is passed (positive/negative, subject/object) is important.
pattern output
apparent meaning
real risk
"X occurs 47 times"
important person
Incomplete collection, spelling variation
"Theme 30%"
dominant topic
sampling bias
"A-B often together"
intimate relationship
Coincidence, missing context
"timeline"
exact chronology
Undated documents, fabricated order
three mini cases
Case 1 — Network analysis revealed a hidden intermediary. A researcher examined a correspondence collection of 800 letters. With AI, who wrote to whom was determined and a network was created. The network showed that a seemingly insignificant person at first glance was actually the key point of contact between the two groups. The researcher focused on this person's letters and reached a new finding. But first verified all the links in the documents.
Case 2 — Normalization error corrupted the number. A student had them count the number of times a place appeared in the documents. Since the AI counted three different spellings of the same place separately, the number turned out to be one third of the real number. When the spellings were combined, place was actually in the third most frequently occurring position. Lesson: without normalization the number cannot be trusted.
Case 3 — Made-up timeline. A researcher created a timeline from undated documents. AI arranged the unknown events in a "logical" order and presented a precise chronology. However, this sequence was not in the documents, AI had made it up. Lesson: the timeline is constructed only from events that have a date in the document; the rest remains "undated".
Four copyable templates
1) NER (entity inference):
The persons, places, institutions and dates mentioned in the documents below appear as SEPARATE lists. Indicate in which document (reference) each entity is mentioned. Take only what is CLEARLY stated in the text; add by guess. Mark [?] if you are not sure of the pronunciation. Documents: [here]
2) Entity normalization:
In the entity list below, group different spellings that could be the same person/place (e.g. "Istanbul / Kostantiniyye"). Suggest the possible common format for each group BUT do not impose the exact combination; Add note "maybe the same, let people verify". Leave it alone if you are not sure. List: [here]
3) Relationship/network outline:
Extract the "who is related to whom" links from the following set of correspondence/documents. For each link, indicate which document (reference) it is based on. MADE up a relationship that is not CLEARLY in the document. Mark ambiguous links [indistinct]. Documentation: [here]
4) Dated timeline:
List in chronological order only the events CLEARLY stated in the following documents; Link every event to its source. List the events with uncertain dates separately under the heading "undated", DO NOT put them in order. DO NOT make up the estimated date. Documentation: [here]
Weak prompt / Strong prompt
Weak:
Analyze these documents and extract important patterns, relationships, and timelines.
“Extract important patterns” pushes AI to invent non-existent connections and precise chronologies, presenting numbers without normalization.
Strong:
Extracts the person/place/institution entities from the following documents by connecting each to the document reference. Mark different spellings that may be the same as "may be merged", but do not merge. Relationships that are not clearly stated in the document and events with uncertain dates DO NOT FAKE; keep those without dates separate. Documentation: [here]
The difference: attribution to source, leaving normalization to humans, ban on fictitious ties/history, and separation of undated events make the pattern defensible.
Common mistakes
- Mistaking the pattern for evidence. The pattern is the hypothesis; is verified in the document.
- Skipping normalization. Different spellings of the same entity distort the number and the network.
- Blind trust in numbers. Incomplete collection and sampling bias make the number misleading.
- Made-up timeline. Undated events are not included in chronology.
- Ignoring context. It is not "how many times it was passed", but "how it was passed" that is important.
In summary
Content analysis and pattern discovery is a powerful field where AI makes visible structures that would not be visible through reading alone: entity extraction, relationship networks, topic clusters, timelines. But every pattern is a hypothesis, not evidence. Link assets to the source, normalize carefully and with human validation, query numbers for missing data and bias, avoid contrived relationships and chronologies. AI marks the pattern; What it means and whether it is true is the historian's judgment.
Application task
Collect at least 15 pieces of documentation (with references). Extract person and place entities with the "NER" template. Then group the different spellings with "entity normalization" and verify the groups yourself. Finally, run the “dated timeline” template and see how many events remain “undated.” Compare how many concocted links or chronologies the AI would produce with poor prompting.
checklist
- [ ] I have clearly defined my pattern question.
- [ ] I have each entity linked to the document reference.
- [ ] I normalized entity typing and combined it with human approval.
- [ ] I questioned the numbers for missing data and bias.
- [ ] I did not put the undated events into chronology, I kept them separately.