Gains:
- Ability to structure the clinical question and conduct literature review, summary and evidence compilation with artificial intelligence
- Ability to manage the risk of fabricated references (hallucinations) by confirming each source given by artificial intelligence with DOI and the original article
- Ability to critically read the AI abstract, assessing the level of evidence, conflict of interest, and timeliness
Pharmacy is a profession whose knowledge is constantly renewed; A dose or recommendation that was known correctly yesterday may change with a new study. Evidence-based pharmacy means answering a question with the most current and reliable scientific evidence. But the scientific literature is huge and scattered; It's easy to get overwhelmed by hundreds of articles when looking for an answer to a question. Artificial intelligence is a powerful scanning and summarizing aid here: it helps structure a question, generate search terms, summarize a long text. But the most dangerous mistake of artificial intelligence is precisely in this area: it can make up articles and DOIs that do not exist. In this unit, you will learn to use artificial intelligence as a safe, validated tool in literature review.
Steps of the evidence-based approach
An evidence-based question is answered with these steps:
- Structure the question. A good clinical question is often framed with the PICO framework: Patient/Population (Patient), Intervention, Comparison, Outcome. Example: “How does drug X (I) affect the risk of hypoglycemia (O) compared to drug Y (C) in elderly people with type 2 diabetes (P)?”
- Look for evidence. Search appropriate databases and resources.
- Evaluate the evidence. Type of study, quality, sample size and risk of bias.
- Apply and monitor. Adapt the finding to the patient's context.
The AI is particularly fast at steps 1 and 3 (structuring the question and helping evaluate a text); but step 2 (sourcing) requires critical verification.
hierarchy of evidence
Not all evidence is equal. A general hierarchy:
Level of evidence
example
power
highest
Systematic review, meta-analysis
Combination of multiple studies
high
Randomized controlled trial (RCT)
Controlled, low bias
medium
Cohort, case-control
Observational, risk of confounding factors
low
Case series, expert opinion
Limited generalizability
lowest
Anecdotal, unverified web content
Poor evidentiary value
When summarizing a finding, AI may confuse the level of evidence or present a weak source as strong. You always control the level.
Additionally, there may be conflicting studies on the same subject. AI can sometimes only highlight one side and ignore the other; this is called "selective presentation". A robust evidence assessment looks for the general trend in the topic (what many studies have in common) and a current systematic review if available. Changing practice based on a single study, especially a small-sample or single-center study, is risky. Asking the AI “are there conflicting findings on this issue, which side has stronger evidence” is a good way to balance a one-sided summary; But the final weighing is done by you by reading the original sources.
Caution: AI may produce completely fabricated references with realistic title, author name, journal and DOI; this is called a "reference hallucination". Never cite a source without opening the DOI in the original database/journal and viewing its content.
three mini cases
Case 1 — Made-up DOI. A pharmacist asked AI for a source on the effectiveness of a drug; artificial intelligence yielded three articles and DOI. When the pharmacist tried to open the DOIs, he found that two did not correspond to any records; one was real but reported a different result. The pharmacist only used the actual article that he had verified. Without verification, two fictitious sources would be included in one report.
Case 2 — Search with PICO. A pharmacist was researching the safety comparison of two painkillers in the elderly. He asked the AI to dump the question into PICO and generate search terms. AI gave clear terms and synonyms; With these, the pharmacist himself performed the actual search in the well-known database and evaluated the results. Artificial intelligence has accelerated search; The pharmacist found the sources and read them.
Case 3 — Summary distortion. A pharmacist had a real article summarized by artificial intelligence. The abstract claimed that the study showed “unequivocal superiority”; but the article itself said the result "did not reach statistical significance." The pharmacist read the original text, noticed the distortion, and corrected the summary. The AI summary had exaggerated; Returning to the source revealed the truth.
Case 4 — Missing the topic. A pharmacist received information from artificial intelligence about the recommended use of a medicine. The answer seemed reasonable, but was based on a guide from a few years ago; However, the recommendation has since been updated. When the pharmacist opened the updated guide, he saw the difference and took the new recommendation as a basis. Because the AI's knowledge was frozen at a specific date, it could not know the most current change; up-to-dateness checking closed this gap.
Evidence-based screening step by step
- Frame the question with PICO. Population, intervention, comparison, outcome.
- Generate search terms. Get keywords and synonyms from artificial intelligence.
- You make the real call. Search reputed database/source; "Making" resources to AI.
- Verify every source. Open DOI, is the author/journal/year real, does the content support the claim?
- Assess the level of evidence. Type of work, quality, conflict of interest, timeliness.
- Contextualize and save. Link finding to patient/question, record sources.
Weak prompt / Strong prompt
Weak: "Is this medicine effective, give a source."
Strong: "Structure the following clinical question in the PICO framework: [question]. Then generate effective search terms and synonyms for that question. DO NOT make up the source/article; tell me what type of studies (RCT, meta-analysis) and which databases I should look for instead. If you name a source, tag 'not verified, must be confirmed with DOI'."
In the strong prompt, the framework, boundary and verification condition are clear; AI is not forced to fabricate resources.
Four copyable templates
Task: Plot the clinical question into PICO.Question: [...]. Output: P (population), I (interference), C (comparison), O (outcome), and a one-sentence structured research question.
Task: Generate search term (FITTING source).Subject: [...]. Output: keywords + synonyms + suggested study types. Credit/DOI; Just tell me how to call.
Task: Summarize this text impartially (no exaggeration).Text: [article]. Output: purpose, method, sample, main finding, constraints, conclusion. Expressing the result STRONGER than what the text says; keep the uncertainty.
Task: Produce reference verification checklist.Reference: [...]. Sort by: DOI opening? Does the author/journal/year hold? Does the content support the claim? What is the level of evidence? Is there a conflict of interest?
Common mistakes
- Quoting the DOI given by artificial intelligence without opening it. The risk of reference hallucination is the greatest danger.
- Substituting the summary for the source. For a critical claim, one should return to the original text.
- Confusing the level of evidence. Expert opinion and meta-analysis do not have the same weight.
- Skipping the update. An older study may conflict with current guidance; The date should be checked.
- Overlooking conflict of interest. Funding and declaration of interest influence interpretation.
In summary
Artificial intelligence is a powerful accelerator for structuring questions, generating search terms, and summarizing long texts in literature searches. But its most dangerous error, referential hallucination, occurs in this area: it can produce sources that appear realistic but do not exist. So do the actual searching yourself, verify each source with the DOI, compare the abstracts with the original text, and critically evaluate the level of evidence. The final interpretation of the evidence and its adaptation to the patient lies with the pharmacist.
Application task
Choose a clinical question from your own practice and have the AI translate it into PICO. Take the search terms and do the actual search yourself on a recognized database. Ask the AI for a few "sources" on the topic and try to open the DOI of each; Note how many are real and how many are made up. Finally, have the AI summarize a real article, compare the summary with the original result, and flag any distortions.
checklist
- [ ] I structured the question around PICO.
- [ ] I took the search terms, did the actual search myself.
- [ ] I opened each source with DOI and verified its authenticity.
- [ ] I compared the summaries with the original text.
- [ ] I evaluated the level of evidence and timeliness.
- [ ] I have considered conflict of interest.
- [ ] I have saved verified sources.