Units
1. Introduction to Artificial Intelligence in Chemistry: Roles, Boundaries, Validation and Ethics 2. Molecular Representation and Chemical Structure: Validation with SMILES, InChI and RDKit 3. Reaction Prediction and Mechanism: Product, Condition and Adverse Reaction Prediction 4. Spectrum Interpretation Support: NMR, IR and Mass Spectrometry 5. Literature Review and Synthesis Planning: Source, Procedure and Roadmap 6. Chemical Data Analysis and Python: pandas, Stoichiometry and Calibration 7. Molecular Property Prediction and QSAR: Resolution, pKa and Model Limits 8. Laboratory Documentation: Experiment Notebook, ELN and Reproducibility 9. Safety and Hazard Assessment: GHS, SDS and the Limits of Artificial Intelligence 10. Confirmation, Hallucination, and Scientific Integrity 11. Human in Experimentation: Ethics, Privacy, Responsibility and End-to-End Workflow
Unit 10 / 11

Confirmation, Hallucination, and Scientific Integrity

Gains:

  • Ability to understand that hallucination is inherent in the language model and choose the appropriate verification method for each type of output.
  • Verify critical values in two independent ways with triangulation and not confuse fluency with accuracy
  • Ability to apply transparency, traceability, data integrity and human responsibility as the basis of scientific integrity

A single phrase returned over and over again throughout this module: “AI produces, humans verify.” In this unit, we turn that sentence into a system. The aim is to recognize hallucination (producing false/fabricated information with a confident language), which is the most dangerous behavior of AI, to establish a concrete verification method for each type of output, and to place all of these within the framework of scientific integrity (honesty, transparency, traceability). An unverified AI output in chemistry is not just a mistake; It could be an experiment exploding, a person being poisoned, or a paper being retracted.

Why is hallucination inevitable?

The large language model produces the “next most likely word”; It aims at what is possible, not what is true. So he can fill in a molecular weight, a DOI, or a boiling point with a value that “seems reasonable” — without checking whether it is real. The hallucination is not a malfunction, but a consequence of the nature of this technology. The solution is not to "try to fix" the model, but to build a validation shell around each output.

The most insidious aspect of the hallucination is its confident language. An incorrect molecular weight is presented with exactly the same confidence as a correct one. So "looks confident" never means "correct."

Attention: How fluent and confident the AI ​​speaks on a subject is not related to the accuracy of that information. Fluency convinces; Only the independent source determines accuracy.

Validation by output type in chemistry

Each output type has its own verification method. Instead of memorizing a single rule, use the appropriate tool for the output:

Output type

Verification method

vehicle

Molecular weight, formula

Recalculate from formula

RDKit / manually

SMILES/structure

Parse + canonicalize

RDKit

Citation/DOI

Solve in database

doi.org, Crossref

Physical constant (kn, pKa)

Reference database

PubChem, handbook

numerical calculation

Write+run code

Python

reaction/condition

Literature confirmation

Reaxys, article

security claim

SDS/GHS

Official SDS

Spectrum assignment

Multi-technical + human

Experimental spectrum

This table is actually a summary of this module: each unit is one row.

Triangulation: three independent paths

The most powerful verification technique is "triangulation": arriving at the same conclusion in three independent ways. For example, for a molecular weight: (1) what YZ says, (2) RDKit's calculation, (3) calculation by hand from the formula. If the three overlap, trust is high; If someone is straying, stop and investigate. Never rely on a single source in chemistry; Look for at least two independent paths.

Step by step: verification workflow

  1. Classify the output: Is it a number, structure, attribution, security assertion?
  2. Select appropriate tool: Get the appropriate verification path for the output type from the table above.
  3. Calculate/search independently: Run the vehicle independently of the AI; compare the result.
  4. If there is a deviation, stop: If the values ​​do not match, do not proceed without investigating which one is correct.
  5. Leave your mark: Record what you verified and how (scientific integrity).
  6. Specify uncertainty: Mark something that cannot be verified as "unconfirmed", do not hide it.

Four copyable templates

1) Self-verification:

You just gave the following output: [OUTPUT]. Task: List each numerical/factual assertion of this output on a separate line. For each assertion: "how to independently verify it?" write method(which tool/database). DO NOT CLAIM that the claim is true; just give a way to verify it.

2) Triangulation control:

I want to check the weight of the following molecule in three ways: [SMILES].Task: 1) Derive the molecular formula.2) Show the molecular weight calculation atom by atom (contribution of each element).Note: I will also calculate it in RDKit and thirdly compare it with the one you gave. If the three do not overlap, warn.

3) Ambiguity marking:

There are multiple factual claims in the following text:[TEXT]Task: Put a confidence level label next to each claim:[VERIFIED] / [VERIFIED] / [UNCERTAIN].Do not mark anything [VERIFIED] if you are not sure.

4) Consistency cross query:

I will ask the same question in two different ways; See if your answers are consistent. Question A: What is the molecular formula of compound

Weak prompt / Strong prompt

Weak:

Write down everything you know about this compound.

AI indiscriminately mixes verifiable and fabricated information; It is unclear which one is real.

Strong:

Provide information about this compound, but label each claim as:[DETERMINISTIC - verified by instrument] e.g. molecular formula,[EXPERIMENTAL - citation needed] e.g. melting point,[UNCERTAIN - not sure].Do not write any unlabeled claims.

Difference: We forced the AI ​​to mark its own uncertainty; We made the verification work plannable.

Scientific integrity: honesty and traceability

Verification is not just a technical step, it is an ethical issue. Scientific integrity requires:

  • Transparency: Don't hide that you're using AI; Specify according to your publication/report policy.
  • Traceability: Record what output came from the AI ​​and how it was verified.
  • Data integrity: “Beautifying” results with AI, discarding outliers without justification is fraud.
  • Responsibility: You are responsible for the final outcome; “The AI ​​said so” is not an excuse.
Tip: Accept from the outset that you can never use the phrase "The AI ​​said so" in a report or defense. This acceptance automatically brings the habit of verifying every output.

mini cases

Case 1 — Triangulation caught an error. A student checked a molecular weight three ways: YZ said 178.2, manual calculation gave 180.16, RDKit gave 180.16. Two independent paths overlapped and eliminated AI. Lesson: two independent verifications silently defeat one false output.

Case 2 — Safe language, wrong information. AI wrote very confidently that a reaction "runs spontaneously at room temperature." The literature showed that this required heating and catalyst. The student trusted the source, not the safe language. Lesson: fluency is not accuracy.

Case 3 — Gained transparency. In the method section of his paper, one researcher clearly stated in which steps he used AI (text editing, code drafting) and wrote that he independently verified all numerical results. The referee process went smoothly. Lesson: transparency builds trust, concealment creates risk.

Common mistakes

  • Trusting the safe language. The AI's confident tone is not an indication of accuracy.
  • Relying on a single source. Accepting a single output without triangulation.
  • Not separating the output type. Thinking that I "checked" the number, attribution, and security assertion with the same method.
  • Hiding uncertainty. Reporting something that cannot be verified as if it were certain.
  • Hiding the use of AI. Lack of transparency undermines scientific integrity.
  • Delegating responsibility. The "AI said so" excuse is invalid; The responsibility lies with the person.

In summary

  • Hallucination is inherent in the language model; The solution is to build a validation shell around each output.
  • Each output type has its own verification method; classify the output and use the appropriate tool.
  • Triangulation (two independent paths) is the strongest control; Don't trust a single source.
  • Fluency and confident language are not related to accuracy.
  • Scientific integrity requires transparency, traceability, data integrity and human responsibility.

Application task

Intentionally produce a variety of outputs in an AI session: a molecular weight, a reference, a physical constant, a security claim, a numerical calculation. Verify each independently with the correct method (see table). Try to catch at least one hallucination. Then prepare a "validation matrix": rows outputs, columns (AI value / independent value / did it overlap / how did I verify it). How would you report this session in terms of scientific integrity?

checklist

  • [ ] I understand why hallucination is inevitable.
  • [ ] I classify each output according to its type and verify it with the appropriate method.
  • [ ] I do triangulation (two independent paths) on critical values.
  • [ ] I trust the source, not the safe language.
  • [ ] I mark what cannot be confirmed as "uncertain".
  • [ ] I am transparent in my use of AI and hold myself accountable for the outcome.
Previous Next