Gains:
- Ability to apply time series, threshold crossing and sensor quality control (QA/QC) concepts
- Ability to create anomaly scanning, trend summary and missing data strategy with AI
- Ability to verify exceedances and anomalies pointed out by AI with raw data and calibration
You are a shift engineer at the wastewater treatment plant of an organized industrial zone. A continuous monitoring station located at the outlet records dissolved oxygen (DO), pH, conductivity and chemical oxygen demand (COD) every 15 minutes. At 03:40 in the morning, the COD value on the SCADA screen jumps to 1,240 mg/L and the system raises an alarm. Discharge limit 250 mg/L. The question you need to ask before you panic is: Is this a real pollution incident or is it a sensor recording? In this unit, you will learn how to use a language model (LLM) as a "first reading assistant" for scanning time series data, marking anomalies and generating trend summaries; But we are discussing why he will never leave the final decision to her.
Anatomy of continuous monitoring data
Environmental monitoring data is a time series collected at regular time intervals. Each measurement point carries a timestamp, a value and ideally a quality flag. To make sense of raw data, you must distinguish three concepts:
- Trend: Long-term trend (e.g. seasonal increase in conductivity).
- Seasonality/cycle: Patterns that repeat during the day or year (e.g., elevation of DO by daytime photosynthesis).
- Anomaly: Singular or short-term values that statistically deviate from the expected pattern.
Parameter
Typical monitoring range
Sample limit (discharge)
Common sensor problem
pH
1-5 min
6-9 (unitless)
Reference electrode shift
Dissolved oxygen (DO)
5-15 min
≥ 5 mg/L (receiving medium)
Membrane fouling (biofouling)
conductivity
5-15 min
Class dependent, µS/cm
Calibration drift
COD (continuous analyzer)
15-60 min
250mg/L
Reagent exhaustion, clogging
PM10 (air)
1 hour
50 µg/m³ (24 hours)
Humidity/condensation effect
Why are QA/QC flags vital?
QA/QC (quality assurance / quality control) is about attaching a trust label to every measurement. Even if a value appears statistically to be an "overshoot", it is invalid if it was taken outside the calibration window or if the sensor is under maintenance. Common flag codes:
Flag Meaning----- -----------------------------G Good — valid, accepted measurementS Suspect — doubtful, verification requiredM Missing — missing / empty recordC Calibration — calibration/maintenance window, data invalidE Estimated — imputed estimated value
Tip: Always open the calibration and maintenance logs before starting an exceedance analysis. If the COD jump at 03:40 coincides with automatic overnight calibration or reagent change, this is a "C" flag data; It is not a subject for alarm, but for data cleaning.
Anomaly scanning and trend summary with AI
You can use LLM to quickly review tabular data, flag patterns that the human eye might miss, and sort out initial hypotheses. The critical point: Giving the model the raw data, units and limit together and asking for verifiable observations from it.
WEAK PROMPT: "Is there a problem with this water quality data?" STRONG PROMPT: "Below is 15 minutes of monitoring data of a wastewater discharge point (columns: time, COD [mg/L], conductivity [µS/cm], pH, QA flag). Discharge COD limit is 250 mg/L, pH range 6-9. Your task: 1) List timestamps with limit exceeds. 2) Only Evaluate lines with 'G' (good) flags; separate those with C/M/S flags into a separate 'verification required' list. 3) Indicate for each overshoot whether it is a singular jump (single line) or a continuous rise (>= 3 consecutive lines). 4) Note whether conductivity and pH change simultaneously (simultaneous change is evidence in favor of the actual event). Do not add any comments that you are unsure of or that cannot be verified with data; write 'must be confirmed with raw data' instead."
The powerful prompt binds the model to a concrete procedure, parses the flags, and includes an explicit "if you're not sure, don't say" instruction to limit hallucination (made-up interpretation).
Caution: LLM may make errors in numerical threshold comparisons; May mislabel 248 mg/L as “exceed” or 252 mg/L as “within limit.” Re-verify EVERY exceedance the model lists, either visually from the raw table or with a formula (e.g. value > 250). The model scans; You decide the limit.
Missing data strategy (imputation)
Sensors become clogged, power goes out, communication breaks down. How you fill in missing data directly affects your trend and exceedance analysis. False imputation can "create" an overshoot that does not exist or hide a real overshoot.
import pandas as pdimport numpy as np# 15 minute COD series; -999 device code "no data" "flag": ["G", "G", "M", "M", "G", "S", "G", "G"],})# 1) Convert device codes to actual missing valuedf.loc[df["koi"] == -999, "koi"] = np.nan# 2) Linear interpolation for SHORT gap (<= 2 steps); flag imputationgap = df["koi"].isna()df["koi_full"] = df["koi"].interpolate(limit=2)df.loc[gap & df["koi_full"].notna(), "flag"] = "E" # Estimated# 3) Limit exceedance scanning — Assignments for decision on measured (G) values ONLY = df[(df["koi_full"] > 250) & (df["flag"] == "G")]print(asyms[["ts", "koi_full", "flag"]])
In this approach, short gaps are filled, but the filled values are marked with E and the overshoot decision is made only from the actual measurement (G). Since the value of 1240 mg/L is flagged as S (suspicious), it is not included in the automatic exceedance list; is verified first.
Verification by sensor drift and calibration
The gold standard for determining whether an overshoot is real or sensor drift is the laboratory result of a concurrent grab sample. For example, if the continuous analyzer showed 1240 mg/L at 03:40, and the laboratory measured value of the sample taken at that time is 205 mg/L, the problem is in the sensor; not discharge. This is triangulation of the AI output or SCADA alarm with the field and laboratory.
mini case
The turbidity sensor in a drinking water basin was showing a gradually increasing trend for three days. The shift team gave weekly data to LLM; “A real increase in turbidity due to post-rainfall runoff is possible,” the model said. However, when the engineer looked at the meteorological records, he saw that there was no rain in the region that week. During the field inspection, it was understood that biofouling had formed on the sensor optical window; After cleaning and calibration, the values returned to normal. LLM's interpretation of the "possible actual event" was refuted by external data (rainfall record) and field control. Lesson: The model may produce a plausible but false story; the verification chain catches it.
Common mistakes
- Mistaking the data in the calibration/maintenance window (flag C) for real exceedance.
- Filling in missing data silently and reporting imputed values as real measurements.
- Mistaking a singular bounce (single line, possibly electrical noise) for a continuous event.
- Blindly trusting LLM's breakpoint comparisons (248 vs 250).
- Making decisions based on a single parameter without cross-checking simultaneous parameters (pH + conductivity + COD).
- Declaring the alarm "real" without ruling out sensor drift with grab sample/laboratory confirmation.
- Ignoring local time/UTC and summer time shifts and attributing events to the wrong time.
In summary
- Environmental monitoring data is a time series; It cannot be interpreted without distinguishing trend, seasonality and anomaly.
- QA/QC flags are the trust tag of the data; decisions are made only from valid (G) data.
- LLM is a quick first read assistant for anomaly scanning, exceedance listing and trend summary, not a decision maker.
- The missing data strategy (imputation) should be transparent; The filled values are marked and are not taken as basis for the exceedance decision.
- Sensor drift, biofouling, and calibration drift produce “spurious overshoot”; distinguishes grab sample and laboratory confirmation.
- AI output is a hypothesis; Raw data does not enter the official report without verification by calibration record and field control.
Application task
Take a 24-hour, 15-minute monitoring dataset you have (or synthetically generate) (at least one parameter: COD, pH, or PM10) and create a run that adds QA/QC flags and lists limit exceedances. First, write a strong prompt and ask LLM for anomaly and exceedance scanning; Then independently verify the same exceedances with the value > limit filter in Python. Make at least one imputation example and mark the filled values with E. For each "overshoot" you find "how do I verify this with the grab sample/calibration record?" Answer the question in one sentence. Ultimately, tabulate how many of the anomalies flagged by LLM are real events and how many are sensor/data problems, with justifications.