Gains:
- Ability to establish a repeatable analysis workflow that loads and cleans telemetry, test and production data with pandas
- Ability to understand and verify the output line by line while receiving code, attributes and visualization assistance from artificial intelligence
- Ability to audit analysis results for unit consistency, data leakage and reproducibility
In previous units, we used artificial intelligence like an "advisor". In this unit, we go one step further and establish a workflow that analyzes automotive data with our own hands, using Python. The goal is not to make you a software developer; It is to make an engineer who can understand and verify that code line by line while getting code assistance from AI. Because the code produced by AI can be faulty, just like the text produced by AI; may confuse volume, leak data, center wrong column. Running the code without understanding it is like publishing an unsigned report.
Python why? Because it is the de facto standard in data analysis: you can easily process telemetry, test and production data with pandas (tabular data), numpy (numerical processing), matplotlib (plot) and scikit-learn (machine learning) libraries.
Six steps to reproducible analysis
A solid analysis always follows the same framework:
- Load: Read data from the file.
- Inspect: Dimension, columns, data types, missing values.
- Clean: Missing/outlier, unit, timestamp correction.
- Extract feature: Derive meaningful variables.
- Analysis/model: Statistics, visualization, or model.
- Verify and report: Result provision, volume/leak check, recording.
Tip: When asking the AI for code, say “comment out what you are doing at each step and write your assumptions.” So you can verify the code as you read it.
Step-by-step example: telemetry analysis
The following code loads a CAN/telemetry record and does basic checking. (Note: make sure you understand the codes before running them; column names vary depending on your data.)
import pandas as pd# 1) Load: semi-dotted CSV, first column timestampdf = pd.read_csv("telemetry.csv", parse_dates=["time"])# 2) Checkprint(df.shape) # how many rows, how many columnsprint(df.dtypes) # type of each column (number or text)print(df.isna().sum()) # number of missing values per columnprint(df.describe()) # summary statistics: min, max, average
The describe() output is your first opportunity to verify: if the engine speed max value is 45,000 rpm (typical passenger engine ~7,000 rpm), there is a unit or sensor fault.
The most frequently used pandas commands in the auditing step and what they do:
command
What does
What does it confirm?
df.shape
Number of rows/columns
Is it the expected size?
df.dtypes
Column types
Has the number column become text?
df.isna().sum()
Missing value count
How much space is there?
df.describe()
Min/max/average
Is it physically reasonable?
df.duplicated().sum()
duplicate row
Is there dual registration?
# 3) Clear: mark physically impossible values
Caution: Do not delete the outlier immediately; First understand why it's an outlier. Is it a real event (sudden heating) or sensor failure? Blindly deleting may hide the real fault.
# 4) Extract feature: temperature rise rate (derivative)df = df.sort_values("time")df["sic_increase_speed"] = df["motor_sic"].diff() / df["time"].diff().dt.total_seconds()# 5) Simple visualizationimport matplotlib.pyplot as pltplt.plot(df["time"], df["motor_sic"])plt.xlabel("Time"); plt.ylabel("Engine temperature (C)")plt.title("Engine temperature course")plt.show()
Preventing data leakage in Python
The most dangerous mistake when building a prediction model is data leakage (unit 5): when the model sees information that cannot be known at the time of prediction. The golden rule to avoid this in time series: train with the past, test with the future — no random shuffling.
from sklearn.model_selection import train_test_split# FALSE: randomly splitting the time series leaks the future# df[df["time"] > threshold] # last 20% future
Caution: train_test_split shuffles the data by default (shuffle=True). In the time series, this confuses the future with education and produces a false high score. If AI has done this in the code it produces, be sure to fix it.
Unit consistency: silent killer
The most insidious errors in automotive are unit errors: km/h to m/s, Nm to lb-ft, bar to kPa, °C to K. Keeping a dictionary of what the unit of each column is in the analysis and writing down the conversions clearly saves lives.
# Document the units explicitly = {"speed": "km/h", "motor_sic": "C", "pressure": "bar"}# Explicit conversion if necessary (km/h -> m/s)df["speed_ms"] = df["speed"] / 3.6
Mini case studies
Case 1 - Error caught with describe(). An analyst runs the code from the AI and makes a range estimate; The result is absurdly high. When we look at the describe() output, we see that the battery capacity is entered in Wh in some lines and in kWh in others (1000 times difference). When the unit is brought to a uniform type, the result improves. Conclusion: A simple summary statistic prevented a bad prediction.
Case 2 - Confusion trap. An intern's brake wear model was 98% accurate on the test set. When the code is examined, it can be seen that train_test_split(shuffle=True) and the time series are mixed, meaning that future points leak into the training. When divided by time, the accuracy drops to 80%, but it is now realistic. Conclusion: Just because the AI code worked, it wasn't right; Human caught the leak.
Case 3 - Understanding the outlier. In a durability record, the AI code automatically deletes outlier strain values. The engineer looks at the deleted points; These were exactly the moments of crack initiation that the test was interested in. Deletion is removed and violations are taken into separate review. Result: Under "Cleaning" the real signal can be deleted; question every step.
prompt templates
Template 1 - Request code (with explanation):
Role: You are a Python data analyst mentor.Task: Write code that loads and checks a telemetry CSV.Context: Columns: time, speed(km/h), engine_sic(C), revolution(rpm).Constraint: Comment out each row; Explain which check was made and why; add physically impossible value check; silently deleting nothing.Output: Commented code + which output I should look at and why.
Template 2 - Leak inspection:
Role: You are a machine learning code reviewer. Task: Check the following training/test split code for data leaks. Context: This is a time series (vehicle telemetry). Constraint: Warn if there is random shuffling; Suggest and justify splitting by time. Output: Findings + corrected code + explanation.
Template 3 - Unit validation:
Role: You are a data quality auditor.Task: Suggest checks that look for unit inconsistency in a DataFrame.Context: Capacity may be kWh in some rows and Wh in others.Output: Check code + how to find the suspicious pattern.
Template 4 - Visualization + comment:
Role: You are a data visualization expert. Task: Write code that plots the engine temperature course and rate of increase. Constraint: Label the axes with the unit; visually mark the anomaly; do not claim definitive fault when interpreting the graph. Output: Code + what to look for in the graph.
Weak prompt / Strong prompt
Weak prompt:
Build a model with this data.
It is not clear which target, which compartment, which verification; AI can output scrambled, leaky, unit-blind code.
Powerful prompt:
Role: You are a Python ML mentor. Task: Write an initial workflow to predict brake pad wear and demonstrate validation pitfalls. Context: Time series telemetry; target: remaining pad thickness.Constraint: Split time series by time (no shuffling); flag attributes at risk of data leakage; document units;interpret each step; Write down what checks are required before the result goes out into the field. Output: Commented code + verification checklist.
Common mistakes
- Running the code without understanding it. AI code may also be faulty; Read line by line.
- Mixing time series. shuffle=True leaks the future and produces a fake score.
- Blindly delete the outlier. The actual signal (fault onset) can be cleared.
- Not documenting the unit. Errors like km/h vs m/s, Wh vs kWh grow silently.
- Skipping the describe()/check step. Most errors appear in the first summary statistics.
In summary
- Python (pandas, numpy, matplotlib, scikit-learn) is the de facto standard of automotive data analysis.
- Repeatable analysis: load, inspect, clean, feature, analyze, validate and report.
- It is imperative to understand and verify the code that AI produces line by line; Just because it works doesn't mean it's right.
- Maintain past/future distinction in time series; Random shuffling creates data leakage.
- Unit consistency and outlier interpretation are silent but critical points of verification.
Application task
Take a small data set (real or synthetic telemetry). (1) Following the six-step framework, generate the loading and control code with Template 1 and confirm that you understand each line. (2) Find at least one suspicious value in the describe() output and investigate why. (3) In a prediction task, time the training/test split and check the risk of leakage with Template 2. (4) Document the unit of each column you use in a dictionary.
checklist
- [ ] I set up the analysis with a six-step framework.
- [ ] I read and understood the code produced by AI line by line.
- [ ] I looked for suspicious values with describe()/audit.
- [ ] I split the time series by time (no shuffling).
- [ ] I marked the attributes that are at risk of data leakage.
- [ ] I documented the unit of each column and wrote the conversions explicitly.