Gains:
- Ability to build a clear hypothesis, accurate controls, randomization, blinding, and prior power analysis into an experimental design
- Ability to distinguish biological repetition from technical repetition and avoid the error of pseudoreplication
- Ability to prevent p-hacking in the first place by fixing the analysis plan before the experiment (pre-recording)
The most expensive mistake in biology is a poorly designed experiment. Incorrect analysis can be corrected later; But data collected with insufficient number of repetitions, uncontrolled or mixed variables cannot be recovered by any statistics. "Garbage in, garbage out." That's why the experiment must be designed correctly before it starts. Artificial intelligence is a powerful thinking partner at this design stage: it recalls control groups, writes the code that performs power analysis (calculating the number of samples required to capture the sought-after difference), questions confounding variables. But the final design decision is yours with domain knowledge.
In this unit, we will discuss the components of a good experimental design and how to use artificial intelligence in this process.
Components of a good experimental design
- Clear hypothesis: A testable, measurable claim. Like "This drug slows down cell growth."
- Independent and dependent variable: What are you changing (drug dose), what are you measuring (cell count)?
- Control group: Comparison group that received no intervention. You can't know the reason for the difference without it.
- Replication: Biological replication (different individuals/cultures) is different from technical replication (repeat measurement of the same sample); The real power is in biological repetition.
- Randomization: Randomly distributing samples into groups and processing order; reduces bias.
- Blinding: The person making the measurement does not know which group is which; prevents subjective bias.
- Power analysis: Calculating in advance how many samples are needed to capture the expected effect with an 80% probability.
Tip: Give the experimental design to the AI to “critique” it. “What control is missing in this plan, what confounding variable did I miss?” The question reveals gaps that you cannot see when thinking alone.
Power analysis: why and how
A very small sample may make even a real difference seem "insignificant" (low power); A sample that is too large is a waste of resources and may make insignificant differences appear "significant". The correct number is found by power analysis and depends on: expected effect size, accepted error rates (alpha and beta), and variability of the data.
Role: You are an experimental design and biostatistics assistant.Task: Perform a power analysis for an experiment comparing two groups.Write code that calculates the number of samples needed per group with expected effect size Cohen's d = 0.8, alpha = 0.05, power = 0.80.statsmodels and explain the assumptions.
You run the code; for example you will get a result like "26 samples per group". You compare this with your experiment budget and create a realistic plan.
Step by step: designing an experiment
- Write the hypothesis: One sentence, measurable.
- Define variables: Independent, dependent, held constant.
- Identify controls: negative control, positive control, vehicle control.
- List the mixers: Time, batch, temperature, experimenter — how will they be controlled?
- Perform a strength analysis: Calculate the number of repetitions required.
- Plan randomization and blinding.
- Write the analysis plan in advance: Determine which test you will use before the experiment (prevents p-hacking).
Copiable prompt templates
Critique my experimental plan. Find weaknesses in terms of: control group, number of replicates, randomization, blinding, confounders. Suggest a concrete correction for each shortcoming. Plan: [text]
Explain with example the difference between biological repetition and technical repetition. In my experience [explanation] which one should be how many and why?
I want to fix my analysis plan BEFORE the experiment. Yield: [design]. Which statistical test is appropriate, which assumptions should I check, will there be multiple comparisons? Write step by step.
Weak prompt / Strong prompt
Weak: “Design an experiment.”
Strong: "I want to test whether Drug
Difference: In the strong prompt, the hypothesis, variables, measurement time and available controls are clear; The model gives concrete, field-appropriate feedback.
three mini cases
Case 1 — Edge effect: One team experimented in a 96-well plate; The results were strange. Artificial intelligence reminded that the edge wells of the plate behave differently due to evaporation (edge effect) and suggested using the edges as controls and distributing the samples randomly. Consistency increased in repetition.
Case 2 — Pseudoreplication: A student took 10 tissue samples from a single mouse and said "n=10." However, these are 10 technical repetitions; biological n is still 1. When the model explained this distinction, the student redesigned the experiment with 6 mice. Lesson: statistical n is a biological unit.
Case 3 — Insufficient power: One group ran 4 samples per group and missed a true effect (p=0.09). Power analysis showed that 4 samples provided only 35% power. If the model had calculated the correct number of samples in advance, the experiment would either have been set up with sufficient power or would not have started at all. Lesson: calculate power beforehand, not later.
comparison chart
concept
Correct application
frequent error
again
Count biological repeat
think of technical repetition
control
Negative+positive+carrier
uncontrolled experiment
Randomization
Group and order random
Sequential processing
power
Calculate before experiment
don't look back
analysis plan
pin in advance
Selecting tests based on data
Common mistakes
- Skipping the control group: Makes the cause of the difference uninterpretable.
- Pseudo-repetition: Considering technical repetition as biological n produces pseudo-significance.
- Not performing a power analysis: Insufficient or wasteful sampling.
- Ignoring confounders: Effects such as batch, location, time distort the result.
- Choosing the analysis test afterwards: Searching for the test that gives the result you want (p-hacking).
Caution: AI gives you statistics and design suggestions, but it requires domain knowledge of the biological meaning of the experiment and what control is actually needed. No analysis can save a poorly designed experiment; Clarify the design before the experiment begins, preferably with a statistician.
Considering sources of variability
The unsung hero of good design is to think ahead about where variability in data comes from. Biological variability (natural difference between individuals) is inevitable and what you want to study; Technical variability (pipet error, device deviation, day difference) is noise and should be reduced. The way to separate the two is in design: distribute technical variability with randomization and blinding, sample biological variability with sufficient repetition.
It's helpful to start by having the AI list possible sources of variability in your experiment. For example, in a cell culture experiment: passage number (how many times cells were divided), medium batch, incubator location, harvest time. Knowing these sources allows you to decide which ones to keep constant and which ones to randomize.
List all possible sources of variability (biological and technical) in the following experiment; Suggest "keep fixed", "randomize" or "save and add to the model as a variable" for each. Experiment: [description]
Pre-registration: the most powerful tool of integrity
The most powerful form of fixing the analysis plan before the experiment is preregistration: recording your hypothesis, sample size, and analysis plan in a time-stamped location before data collection. This eliminates the later doubt of “I tried analysis until I got what I wanted” and greatly increases the credibility of your work. AI can help you draft your term sheet, clarify the hypothesis, and write the analysis plan. In this way, you will have established the strongest shield against p-hacking, which we will see in Unit 8, before the experiment begins.
Design differences according to experiment type
Not every biology experiment follows the same pattern; The design varies depending on the type of problem. A dose-response experiment requires multiple doses and a curve model; A single dose is not enough. In a time series experiment, the spacing and dependency of measurement points (re-measuring the same individual) determines the design. In a field/ecology study, random sampling and spatial dependence (similarity of nearby points) come to the fore. You tell the AI the type of your experiment and ask “what design elements are critical for this type?” Asking: allows you to catch an overlooked component early. But remember: the model recalls general principles, it does not know the biological details of your system.
In summary
A solid experiment; It is based on a clear hypothesis, accurate controls, sufficient biological replicates, randomization, blinding, and pre-calculated power. AI is a powerful partner that recalls these components, writes the power analysis code, and critiques your plan. But the biological meaning and final design decision is yours. Fixing the analysis plan before the experiment is the antidote to the p-hacking trap we will see in the next unit.
Application task
Describe an experiment you have in mind (or an example) in one paragraph: hypothesis, variables, controls, planned replication. You had the artificial intelligence criticize this plan; List missing controls and mixers. Then print a power analysis code (with your d and alpha values) and have it calculate the number of samples required. Revise your plan with this feedback.
checklist
- [ ] I wrote my hypothesis in one measurable sentence.
- [ ] I determined the necessary control groups (negative/positive/carrier).
- [ ] I distinguished biological and technical repetition correctly.
- [ ] I did the power analysis before the experiment.
- [ ] I planned randomization and blinding.
- [ ] I fixed the analysis test without seeing the data.