Gains:
- Ability to use artificial intelligence in literature mapping, concept scanning and reading prioritization to manage the risk of fabricated sources (hallucinations)
- Ability to establish a verification flow that confirms every citation, finding and effect size claim from the primary source
- Ability to distinguish the level of evidence (meta-analysis, RCT, case) and weigh the AI summary against the hierarchy of evidence
Psychology is an evidence-based profession: research evidence tells us whether an intervention works, not “it just seems like it to me.” When an expert wants to stay current, write a thesis, or choose the right approach for a case, he turns to literature. But the literature is huge; Tens of thousands of articles are published every year. Artificial intelligence (AI) is a tremendous accelerator here: it defines a concept, maps the field, suggests which article you should read first, simplifies a complex text. But herein lies the most dangerous feature of AI in the literature: it fabricates non-existent sources. In this unit, you will learn how to make AI an accelerator in the literature business, how to work without falling into the trap of fabricated sources (hallucinations).
What does AI do in the literature, what does it make up?
Literature tasks where AI is strong: explaining a concept in plain language; summarize major debates in a field; to make a text readable; suggest keywords and search strategy; summarize an article (when you provide the text); comparing different views. In these, AI reduces hours of reading to minutes.
Things AI makes up: citations (sources that look perfect but don't exist, including author, year, journal, DOI number); statistics (real-looking numbers like effect size, sample size, p-value); findings (a study claims to have "found that" when in fact it has not). This is because AI is not a search engine, but a “likely word guesser”: it generates the most likely-looking citation format, without checking its accuracy.
Attention: Any citation given by AI is not valid without you verifying it from the primary source (the article itself). Even a seemingly perfect DOI can be fake. It's always better to say "I couldn't find it" than "I put the wrong source."
Hierarchy of evidence: not all findings are equal
When AI summarizes a finding, it does not tell you its evidentiary strength. However, there is a hierarchy of evidence. When weighing a claim, it is essential to know what type of evidence it is based on.
Level of evidence
What do you mean?
power
Meta-analysis / systematic review
Statistical combination of multiple studies
highest
Randomized controlled trial (RCT)
Experiment in which participants are divided into random groups
high
Longitudinal/cross-sectional study
Observational, non-experimental
medium
Case series / case report
Description of a single or several cases
low
Expert opinion / theoretical writing
More interpretation than evidence
lowest
When AI tells you “research shows…” your first question should be: Which type of research? The finding of a meta-analysis does not carry the same weight as the finding of a single case report.
Step by step: AI-powered secure literature flow
- Have the AI explain the concept. To get to the point quickly, ask the AI for a simple summary and key concepts. (This is a starter, not a source.)
- Set up search strategy. Ask AI to suggest keywords, synonyms, and search string; use it on real databases (academic search engines, journal archives).
- Find real sources yourself. Obtain articles from peer-reviewed databases. Use AI-suggested titles as search clues, not as citations.
- Summarize text to AI. Give the text of the actual article you found to the AI and ask for a summary/comparison. AI no longer makes up stuff, because it has real text.
- Confirm every number and finding. AI compare an effect size in the abstract with the actual value in the text.
- Label the level of evidence. Write the type of evidence next to each finding.
Tip: Use AI in literature in two different modes: “sourceless suggestive” (concept exposition, search idea) and “sourced summarizer” (when you give the text). The latter is safe because the AI works from the text you provide, not from its own memory.
three mini cases
Case 1 — The fake attribution epidemic. A graduate student requests AI citation for a 15-source chapter on anxiety disorders. AI gives it all in APA format. When the student checks, he sees that 6 out of 15 sources do not actually exist (40%) and 3 are assigned to the wrong journal. So half the list is unusable. Lesson: attribution comes from the real database, not the AI.
Case 2 — Correct use. A clinician gives the PDF text of 4 real meta-analyses to AI one by one and says "table the sample number, main effect size and limitation of each one." AI reduces 4 30-page articles into a comparative table in 10 minutes. The clinician confirms each number by text; All of them turn out to be correct because the AI worked from the given text, not from its own memory.
Case 3 — Level of evidence trap. An expert asks the AI, “Does therapy X work?” he asks. The AI says “yes, studies support it” and counts four “studies.” When the expert examines it, he sees that three of them are a single case report and one is a blog post; There is not even a single RCT. Although the claim may seem formally "proven", its evidential power is very weak.
Copiable prompts and templates
I'm just getting started on this topic. WITHOUT CREATING ME SOURCES, just give me a conceptual introduction: basic concepts, major discussions of the field, and a list of keywords + synonyms I can use when searching. Attribution; If attribution is required, say "find the source yourself." Subject: [subject]
I'm pasting the text of an actual article below. Work only from this text, do not add outside information. Extract the following: research question, method, sample (n), main finding and effect size, limitations, level of evidence. Text: [article text]
Below is the text of 3 articles. Put them in a comparative table with the following columns: author-year, sample, method, main finding, level of evidence, limitation. Work only from the given texts, do not add sources. Texts: [texts]
List each numerical claim (n, p, effect size, percentage) in this text on a separate line so that I can confirm it with the original article. Do not add any numbers that you cannot verify yourself.Text: [summary]
Weak prompt / Strong prompt
Weak prompt: "Give 10 academic sources about the effectiveness of cognitive behavioral therapy in depression."
This prompt invites the AI to make up attribution from memory; A significant portion of the output will be spurious.
Powerful prompt: "Suggest keywords, synonyms, and search strings to search the database on cognitive behavioral therapy and depression. Do not provide sources/citations; I will find the articles from the actual database, then give you their texts."
This prompt puts the AI in the safe role (search strategist); The references come from the real source.
Common mistakes
- Directly using the attribution given by the AI. Seemingly perfect sources are often fabricated; It is never used without confirmation.
- Blindly trusting the DOI/link. AI can also produce realistic but non-existent DOI; It will not be considered valid until you open and check it.
- Ignoring the level of evidence. Not asking whether there is a blog or a meta-analysis behind the sentence "studies say".
- Substituting the AI summary for the article. It is a summary map; Basing a critical decision only on the summary means making a judgment without reading the text.
- Using in single mode. Always using AI in "resource from memory" mode; whereas the safe one is the "summarize from the given text" mode.
In summary
In the literature business, AI is a powerful accelerator for concept clarification, search strategy, and summarization (when you provide the text). But he fabricates references, statistics and findings from his own memory (hallucination). Verify each citation and number from the primary source; Consider the source you cannot find invalid. Label the level of evidence (from meta-analysis to case report) for each finding. Use AI in two distinct modes: “unsourced suggestive” and “sourced summarizer,” with a preference for the latter.
Application task
Choose a topic. First ask the AI for 5 citations on that topic (sourced mode). Then search these 5 citations one by one in a real academic search engine and note how many actually exist. Then find 2 real articles on the same topic, feed their texts to the AI, and have it produce a comparative table. Compare two experiences: which one gave reliable output?
checklist
- [ ] I did not use any of the references given by AI without confirming them.
- [ ] I obtained the articles from a real, peer-reviewed database.
- [ ] For the summary, I gave the AI the actual article text (not from memory).
- [ ] I compared each numerical claim to the original text.
- [ ] I labeled the level of evidence for each finding.
- [ ] I invalidated the source that I could not find.