Gains:
- Ability to evaluate nutrition claims by considering the hierarchy of evidence and distinguishing between correlation and causation
- Ability to recognize artificial intelligence's source hallucination and verify each reference from the primary source
- Ability to use artificial intelligence not to find sources, but to question and critically evaluate the source
Nutrition science evolves rapidly and new research is constantly published. A dietitian's professional value depends on his ability to discern the truth in this flood of evidence, stay up to date, and base his claims on a solid foundation. At the same time, nutrition is one of the areas where pseudoscience thrives the most: a new "miracle food", "toxin", "superfood" claim circulates every week. AI can help on both fronts: it summarizes scientific texts, simplifies complex studies, helps you question the level of evidence for a claim. But this is where AI's most dangerous trap lies: it can fabricate studies that don't exist, invent sources, and present weak evidence as strong. The principle of this unit: use AI to evaluate and question evidence, not to find it; verify every source and every claim from the primary source.
Hierarchy of evidence: not all studies are equal
In science, the power of evidence forms a pyramid. Power increases from bottom to top:
Level of evidence
What do you mean?
power
Expert opinion / anecdote
Personal experience, "it worked for a friend of mine"
weakest
Case series
Observation of a small number of patients
weak
observational study
Monitoring population (correlation)
medium
Randomized controlled trial (RCT)
Experiment with random groups (cause and effect)
strong
Systematic review/meta-analysis
A consolidated summary of hard work
strongest
Knowing this hierarchy is critical; Because most "this food does that" claims in the news are actually based on weak observational studies. "Correlation" (two things occurring together) is not "causation" (one causing the other): an observation that coffee drinkers live longer does not prove that coffee extends life. AI sometimes blurs this distinction; You need to clarify.
AI's source hallucination
The most insidious mistake of language models is that they fabricate scientific studies, author names, journal names and DOI numbers that do not actually exist. AI can tell you in a convincing sentence, “According to a study published in the Journal of Nutrition in 2021…”; that work may never exist. So the golden rule: Do not use any source given by YZ without confirming it from the primary source (PubMed, Cochrane, original journal). Using AI as a literature search engine is a professional mistake.
Caution: If you tell AI "list studies that support this" you may get convincing but fake references. Correct usage: give the text of a real study to the AI and ask "summarize this / what are the limitations of this study?" is to ask. You bring the source, AI interprets it.
Trusted guidelines and scientific consensus
A dietitian does not have to scan the literature from scratch for each question; There are national and international guidelines for most clinical situations. Guides are regularly updated consensus documents from which expert panels evaluate and conclude numerous studies; for example, Türkiye Nutrition Guide (TÜBER), World Health Organization (WHO) recommendations, nutrition guidelines of diabetes and cardiology associations. When evaluating a claim, first ask "what does the current guidance say about this?" ' is more reliable than getting lost in individual studies; because the guide has already weighed the body of evidence. You can use AI in two ways here: to summarize the actual text of a guide, or to question whether a claim is consistent with the general consensus in the guides. But be careful: the directory information in the AI's memory may be old or confused; Always confirm with the official source which version of the guide is current and the exact recommendation. Before you say "Science says this," verify the current consensus of that science from the actual document. Guides are updated as the science changes; A recommendation from three years ago may have been revised today.
Using AI correctly in evaluating evidence
- You bring the source. Find an actual article, summary or guide; Give it to the AI.
- Summarize and simplify. Have complex text translated into understandable language — but compare the claims to the text.
- Question the restrictions. “What are the methodological weaknesses of this study?” AI is good at generating critical questions.
- Establish the claim to the level of evidence. Have them classify the level of evidence on which a claim is based.
- Verify from primary source. Verify every issue, every result, and every source with the original.
three mini cases
Case 1 — Fabricated source. For a presentation, a dietitian tells the AI, "give 3 studies that prove the benefits of intermittent fasting." AI gives three studies, author and year. The dietitian searches on PubMed: two of the three do not exist at all, one is on a different topic. If he had included it in the presentation, his professional credibility would have been destroyed. The correct way was to find the actual compilations itself and have the AI summarize them. Lesson: AI invents resources; Verify each reference.
Case 2 — Correlation-causation trap. A news article headlines "breakfast makes you lose weight", based on an observational study in which "those who eat breakfast are thinner." A client asks this. The dietitian has the AI summarize the study and question its constraints; It turns out that people who eat breakfast generally live more actively and regularly, meaning that the relationship is not causal. The dietitian provides balanced information to the client. Lesson: observation is not causation.
Case 3 — Correct use. A dietitian finds a recent meta-analysis of protein needs (real, from PubMed), but the text is long and technical. Give it to the AI and ask “what is the main finding, sample, constraints, and practical implication?” he asks. AI faithfully summarizes the text. The dietitian verifies the summary by comparing it with the original and adapts it to his practice. AI reduced the reading load; source and verification remained with the dietitian.
Copiable prompt templates
STUDY ABSTRACT TEMPLATE (YOU provide the source)Below is the text/summary of an actual scientific paper. Summarize this with the following headings: research question, method/design, sample, main finding, LIMITATIONS, practical implication. Do not add anything that is NOT in the text; fitting. Text: [...]
LEVEL OF EVIDENCE CLASSIFICATION TEMPLATEFor the following claim: "[claim]". Consider the level of evidence on which this claim is typically based (anecdotal / observational / RCT / meta-analysis). State whether there is correlation or causation. Source FAKE; just analyze the evidentiary strength of the claim.
CRITICAL INQUIRY TEMPLATEList the methodological WEAKNESSES and possible biases (small sample size, short duration, conflict of interest, confounding factor) of the following study summary. Do not exaggerate the finding; balanced criticism.Summary: [...]
CLAIM REALITY CHECK TEMPLATEA client heard the following claim: "[e.g., detox tea removes toxins]". Honestly evaluate the scientific basis of this claim: what is known, what is not known, what parts are false/exaggerated. Draft a measured, non-frightening explanation to be told to the client. Source fabrication.
Weak prompt / Strong prompt
Weak prompt:
List studies proving that ginger strengthens immunity.
This desire pushes the AI to fabricate sources and produces bias by asking one way (“prover”). Fake testimonials arrive.
Powerful prompt:
Your role: evidence evaluation assistant; source FAKE Evaluate the "ginger boosts immunity" claim balancedly: what level of evidence is this claim typically based on, is the evidence strong or weak, correlation or causation? Make the distinction between what is known/what is unknown. DO NOT reference actual studies (I will verify them from PubMed); just analyze the evidentiary strength of the claim.
The second prompt prohibits fabrication of sources, requires balanced (rather than one-sided) evaluation, and leaves verification to humans; output is a safe thinking tool.
Common mistakes
- Mistaking AI for a literature search engine: Asking for resources; AI makes up references.
- Not confirming the source: Using the work provided by AI without verifying it from the primary source.
- Mistaking correlation for causality: Presenting observational findings as "this does that".
- Ignoring the level of evidence: Equating anecdote with meta-analysis.
- One-sided asking: Creating bias by saying "proving studies"; balanced problem.
- Neglecting currentness: Assuming old/outdated information is current; See the current version of the guides.
In summary
Reading evidence and paying attention to health claims are the foundation of a dietitian's professional credibility. AI is a powerful reading and inquiry assistant in this field: it summarizes real studies, extracts constraints, analyzes the level of evidence of claims. But AI fabricates sources — this is its most dangerous mistake. Always bring the source and verify it from the primary source; separate correlation from causation; observe the hierarchy of evidence; Question the allegations in a balanced way. AI speeds up your thinking; but the responsibility for the science, the accuracy of the source, and the honesty of the claim are yours.
Application task
Choose a popular nutrition claim (e.g. "alkaline diet balances the body" or "eating late at night makes you gain weight"). Get a balanced assessment from AI with the “claim reality check template”. Then use the "level of evidence classification template" to determine the level of evidence on which the claim is based. Also, knowingly ask AI for a “source list” and search one of the references it gives in PubMed; See if it's real. This experiment will embody the source hallucination.
checklist
- [ ] I brought the source; I didn't make up a reference to the AI.
- [ ] I confirmed every source given by YZ from the primary source.
- [ ] I distinguished between correlation and causation.
- [ ] I have classified the claim according to the hierarchy of evidence.
- [ ] I questioned the claims in a balanced (not one-sided) way.
- [ ] I checked the compatibility of the information with current guides/compilations.