Gains:
- Ability to separate segmentation and classification tasks and choose the appropriate ready-made tool (Cellpose, StarDist, MegaDetector) for each
- Ability to extract reliable measurements from images by applying scale calibration and manual verification
- Understand the need to direct low confidence score predictions to human validation and confirm the model outside the training distribution.
Biology is a visual science: cells under a microscope, colonies in a petri dish, animals in a forest camera, disease spots on a leaf. Counting, measuring and classifying these images used to be a tedious and subjective task that took hours. Artificial intelligence, especially deep learning (pattern recognition with multi-layer artificial neural networks) based image models, has accelerated and standardized this work. In this unit, we will discuss how to use artificial intelligence in biological image analysis, ready-made tools and validation requirements.
Warning from the start: a model may "recognize" a cell or species, but it may also misrecognize; human eyes and real-label verification are essential for critical decisions.
Two types of display tasks
- Segmentation: Separating and counting objects (cells, nuclei) in the image pixel by pixel. Example tools: Cellpose, StarDist. “How many cells are in this microscope image and how big is each?”
- Classification/detection: Telling what is in the image or finding its location. Example: recognizing the species in a camera trap photo (iNaturalist, MegaDetector), classifying whether a leaf has disease or not.
These two tasks require different tools. Artificial intelligence (LLM) helps you choose the right tool, write the installation and invocation code, and interpret the output.
Tip: In image analysis, most of the time you don't need to train a model from scratch. Pretrained tools like Cellpose work directly on many cell types. Try the ready tool first; but confirm the result manually in several images.
Step by step: counting cells on microscope images
- Prepare images: Consistent magnification, illumination; note the file format (TIFF, etc.).
- Scale calibration: Know how many micrometers per pixel (essential for size measurement).
- Select segmentation tool: StarDist if core staining; Cellpose for cytoplasm.
- Run: Try the tool on an image.
- Verify manually: Compare the cells the model finds with the actual image; Count merged/missed cells.
- Batch: If validation is satisfactory, apply to the entire set.
- Report: Number of cells, size distribution; Document method and accuracy.
Copiable prompt templates
Role: You are the bioimage analysis assistant. Task: Write Python code that segments and counts cells in a microscope image with Cellpose. Input: image.tif, single channel. Output: cell count and mask visualization. Give working code with comments and justify model choice.
How do I validate my segmentation output? Suggest a method and metric (precision, recall, F1) to compare the number of cells found by the model with the sample I manually counted. Write code too.
Explain the MegaDetector workflow for detecting animals in my camera trap photos. Explain the time savings and risk of false negatives in eliminating blank frames.
Write code that makes pixel-to-micrometer conversion with scale bar information and calculates the area of each cell in square micrometers. Calibration: [X] pixel = [Y] micrometer.
Weak prompt / Strong prompt
Weak: "Count the cells in this photo."
Strong: "I have a DAPI-stained nuclei image (TIFF, single channel, 2048x2048, scale 0.32 µm/pixel). Write code that segments and counts nuclei with StarDist's pre-trained 2D model, giving the area of each nucleus in square micrometers. Then add code that compares with manual counting in 3 images and produces an accuracy report."
Difference: The powerful prompt has the image type, painting, resolution, scale and verification scheme. The model uses the correct tool and the correct scale.
three mini cases
Case 1 — Confluent cells: A student counted 400 cells in a dense tissue image with Cellpose; When I counted by hand, it turned out to be 560. The model combined the cells touching each other as a single cell. The AI suggested adjusting the cell diameter parameter and changing the flow threshold; The count rose to 545. Lesson: adjust the parameters according to the real image.
Case 2 — Species confusion: In an ecology project, automatic species recognition repeatedly tagged a fox as “cat” in nighttime images. The model explained that low light and imbalance in the training data could cause this error; The team forwarded the ambiguous tags to human validation. Lesson: if the model's confidence score is low, human verification is mandatory.
Case 3 — Scale error: A researcher reported cell areas in pixels but forgot to convert to micrometers; The result was incompatible with the literature. When the artificial intelligence added the calibration step, the values fell within a reasonable range. Lesson: physical unit conversion is critical.
comparison chart
Quest
suitable vehicle
verification metric
Core segmentation
StarDist
F1 with manual counting
Cytoplasm/cell boundary
cellpose
IoU (overlap rate)
Species identification (wildlife)
MegaDetector/iNaturalist
Trust score + human approval
disease classification
Specially trained model
Confusion matrix
Common mistakes
- Batch processing without manual validation: Propagating the model's error across the entire data.
- Skipping scale calibration: Not converting pixel-unit to physical unit.
- Accepting predictions with low confidence scores: Not referring uncertain situations to humans.
- Leaving parameters as default: Not adapting settings such as cell diameter to the image.
- Image outside the training distribution: Blindly using the model in the condition it has never seen (different coloring, type).
Caution: Image models may perform well on images similar to the training data, but unexpectedly poorly under different conditions (new stain, new species, different microscope). When moving to a new data set, do not trust the model without manually validating it on several images. Human consent is essential for high-consequence decisions such as species conservation or diagnosis.
From segmentation to measurement: beyond numbers
Counting cells is often the first step; The intrinsic value extracts measurements about each object. Dozens of features per cell, such as area, perimeter, roundness, fluorescence intensity (how much a marker is expressed), can be calculated from a segmentation mask. The regionprops function in the scikit-image library does this. The AI writes code that extracts these measurements and pours the results into a table; you can also compare groups (e.g. “are the treated cells smaller than the control?”).
Extract the area, perimeter and average fluorescence intensity of each cell from the Cellpose mask with scikit-image regionprops. Write the result to CSV; Compare the control and treatment groups in the box plot. Apply scale calibration (µm/pixel).
Tip: Don't forget to remove the background when measuring density; Uncalibrated intensity values depend on the microscope setting and are comparable not in absolute terms, but in relative terms between groups taken under the same condition.
When to train a model: little data, careful labeling
Sometimes off-the-shelf tools are not enough and you need to train your own classification model (e.g. a specific disease symptom). Two traps stand out here. The first is data leakage: the images of the same individual/sample being included in both the training and test sets make the model seem more successful than it is; do the division at the individual level. The second is the unbalanced class: if the patient samples are few, the model shows high accuracy by saying "always healthy" but is useless; Look at sensitivity (recall) and confusion matrix instead of accuracy. AI writes this training code, but it's your job to navigate these methodological pitfalls.
Image quality and preprocessing
The success of a display model is directly dependent on the quality of the image you provide. Images that are out of focus, overexposed, low in contrast, or taken at different magnifications will confuse even the best model. That's why simple pre-processing steps before segmentation often significantly improve the result: background correction (removing lighting imbalance), contrast normalization and noise reduction. The AI writes this preprocessing code (with scikit-image or OpenCV); But you decide what correction is necessary by looking at the image. Excessive preprocessing is also dangerous: “cleaning” an image too much can erase actual biological structure and turn into data beautification. The rule is this: preprocessing should be applied to all images in the same, predefined manner, not selectively done one by one to beautify the result.
Apply standard preprocessing to my microscope images before segmentation: background correction, contrast normalization, slight noise reduction. Apply the same parameters to ALL images (not selectively). Show before/after comparison. Use scikit-image.
In summary
Artificial intelligence saves a lot of time in biological image analysis (cell segmentation, species/disease detection). Off-the-shelf tools like Cellpose, StarDist, MegaDetector solve most tasks without training from scratch. However, the result must be verified manually, the scale must be calibrated, and low confidence scores must be directed to humans. The model is unreliable outside of the training distribution; Confirmation is essential when moving to new data.
Application task
Take a microscope image (or sample data). Have the artificial intelligence write and run code that segments and counts cells with Cellpose or StarDist. Compare by hand counting the cells the model finds in at least one image; Note how many cells were missed/merged. Report cell areas in square micrometers by adding scale calibration.
checklist
- [ ] I added the image type, painting and resolution to the prompt.
- [ ] I applied scale calibration (pixel-micrometer).
- [ ] I manually verified the result on at least one image.
- [ ] I set the segmentation parameters according to the image.
- [ ] I forwarded predictions with low confidence scores to human validation.
- [ ] I did not trust the model without confirming it in the new data set.