Gains:
- Ability to explore the distribution, center, spread and anomalies of variables with appropriate graphs (histogram, box plot, scatter plot).
- Ability to interpret correlation correctly and read it together with the scatter plot, without confusing it with causality
- Ability to understand that summary statistics can be misleading (Anscombe lesson) and develop hypotheses that determine the direction of modeling.
Before building a model, it is necessary to know the data. Just as a doctor does not prescribe medication without examining the patient, a data scientist does not build a model without "examining" the data. This examination is called exploratory data analysis (EDA for short): it is the phase of discovering the distribution, relationships, surprises and traps of the data visually and graphically. The purpose of EDA is to generate hypotheses, catch errors, and determine modeling direction. Artificial intelligence is a very powerful assistant at this stage: it suggests which chart is suitable, writes its code, interprets a distribution. But a person decides, with his/her field knowledge, whether the pattern he sees is real or a coincidence.
What does EDA look for?
EDA seeks answers to four questions. Distribution: How is each variable distributed—is it symmetrical, skewed, or two-peaked? Central tendency and spread: What are the mean, median, standard deviation? Relationships: How are the variables related to each other? Anomalies: Are there unexpected values, gaps, groupings? These four questions determine which feature will work and which model is appropriate.
Let's define some basic concepts. A histogram is a graph that divides the values of a numerical variable into intervals and shows how many observations are in each interval; It is the fastest way to see the distribution. Skewness is the shift of the distribution in one direction; Variables such as income are skewed to the right (majority low, few very high). A box plot shows the median, quartiles and outliers at a glance. A scatter plot shows the relationship of two numerical variables with a cloud of points.
Correlation: there is a relationship but be careful
Correlation — a measure of how two variables move together; a number between -1 and +1 — is the most used and most misunderstood tool of EDA. +1 means perfect co-increase, -1 means perfect inverse relationship, 0 means no linear relationship. There are two big traps. First, the famous rule: correlation is not causation. Choking cases increase with ice cream sales; not because one leads to the other, but because summer (the hidden third variable) triggers both. Second, correlation only measures linear relationship; It may indicate a strong but curvilinear relationship close to 0. So when looking at correlation, be sure to look at the scatter plot as well.
Caution: When the AI gives a correlation number, it may slip into causal language like “A increases B.” This is dangerous. Correlation only indicates movement together; Only experience, domain knowledge, and careful inference establish the reason.
Anscombe quartet: why not just look at the number
There is a famous example in statistics: the Anscombe quartet. Four different data sets have almost the same mean, the same variance, and the same correlation (0.82); But when graphed, they look completely different — one straight line, one curved, one a cloud pulled by a single outlier. The lesson is clear: summary statistics are misleading, looking at the graph is a must. AI can give you averages and correlations, but trusting those numbers without seeing the graph is falling into the Anscombe trap.
The table below summarizes which chart fits which question:
Question
suitable graphic
What shows
Distribution of a single variable
Histogram / KDE
Shape, skewness, number of vertices
Center and outliers
Box plot
Median, quartiles, outliers
Relationship of two numbers
scatter chart
Direction, strength, curvature
Category comparison
bar chart
Difference between groups
change over time
line chart
Trend, seasonality
Multivariate relationship
Correlation heat map
General relationship matrix
Simpson's paradox: aggregated data can lie
A sneaky trap in EDA to be aware of is Simpson's paradox: a pattern may show one direction when all data are examined together, but reverse when the data is divided into meaningful subgroups. Classic example: the overall acceptance rate for female applicants at a university appears to be lower than for males; But when looked at department by department, the acceptance rate of women is higher than men in most departments. From where? Women applied to more competitive (lower acceptance rates) departments; The combined number stores this hidden variable. The lesson is clear: when looking at a total or average, be sure to check whether that number tells the same story when broken down into meaningful subgroups (segment, period, channel). When the AI gives you a combined rate, asking “is it still valid when you segment it” will catch most misleading results early on.
three mini cases
Case 1 — Two hidden hills. One analyst found the average customer age to be 41 and reported it as a "middle-aged crowd." But the histogram showed two peaks: the 22-28 and 55-62 age groups. The average 41 did not actually represent anyone. Lesson: plot the distribution before trusting the mean.
Case 2 — Spurious correlation. One team found a 0.78 correlation between “ad spend” and “sales” and doubled the budget. However, what triggered both were seasonal campaigns; during the off-campaign period, the relationship dropped to 0.10. 340 thousand TL additional advertising did not yield the expected return. Lesson: mistaking correlation for causation is expensive.
Case 3 — The line drawn by the lone contrarian. A real estate analysis showed a strong relationship between square footage and price (r=0.80). When the scatter plot was drawn, almost the entire relationship was created by a single 12 million TL luxury villa; When that point was removed, the correlation dropped to 0.31. Lesson: always read the correlation along with the scatter plot.
Four copyable templates
1) Automatic EDA summary:
I have pandas df. Write code that produces: (1) df.info() and df.describe(), (2) histogram for each numeric column, (3) count for each categorical column. Don't comment, just generate the discovery code; I interpret the patterns.
2) Correlation + visual (with the caveat of causality):
Write code that draws a correlation matrix and a heat map for the numeric columns. Also draw a scatter plot of the 3 highest correlated pairs. Add a comment line to the output reminding you that correlation does not imply causation. Do not make causal claims.
3) Distribution diagnosis:
For the "income_tl" column: write code that produces histogram, box plot, skewness value and median/mean comparison. I will interpret whether the distribution is skewed or not; just give the code.
4) Segment comparison:
Compare customers by "channel" (web/mobile): write code that produces a boxplot of the average amount, median amount, number of orders, and amount distribution for each channel. Suggest which test is appropriate for statistical significance of the difference between the two channels, but the decision is up to me.
Weak prompt / Strong prompt
Weak prompt:
Find something interesting in this data.
“Interesting” is undefined; AI does not see the data, so it either makes it up or makes general statements. There is no direction, no variables, no purpose.
Powerful prompt:
Your role: EDA assistant. df columns: age (int), income_tl (float), city (category), channel (web/mobile), amount (float). Purpose: to discover the factors affecting "amount". Task: (1) code plotting the distribution of amount, (2) distribution graphs between amount and age and income_tl, (3) box plot of amount in channel breakdown. Establishing a causal claim; Just generate the discovery code and I'll interpret the pattern.
Here, the purpose, variables and the limit of "claiming causality" are clear.
Common mistakes
- Relying solely on summary statistics. As the Anscombe quartet shows, the same mean hides very different distributions; see chart.
- Mistaking correlation for causation. Co-action does not indicate that one leads to the other; Beware of hidden variables.
- Looking at correlation without a scatter plot. A single outlier or curvilinearity misleads the number.
- Summarizing a two-peaked/skewed distribution with the mean. The average may not represent anyone.
- Skipping EDA and going straight to the model. Unrecognized data means blind model.
Tip: With every new data set, spend the first 30 minutes just drawing graphs: histogram of each numeric, scatter plot of significant pairs, bar chart of categories. These 30 minutes prevent future modeling errors for days to come.
In summary
EDA is the phase of getting to know the data before building a model and seeks answers to four questions: distribution, center-spread, relationships and anomalies. Summary statistics can be misleading; looking at the graph is a must (Anscombe lecture). Correlation is a powerful tool, but causation is not and should definitely be read in conjunction with a scatter plot. AI is a great accelerator for suggesting graphics and writing code; But people interpret with their field knowledge whether the pattern they see is real or a coincidence.
Application task
Choose a dataset and just do EDA: extract a histogram of at least three numerical variables, a scatter plot of two pairs of variables, and a correlation heat map. Find at least one “surprise” (such as a distribution with two peaks, a candidate for spurious correlation, a relationship attracted by a single outlier) and explain it in one sentence. Write without making any causal claims.
checklist
- [ ] Have I graphed the distribution of each numerical variable?
- [ ] Did I look at the correlations with the scatterplot?
- [ ] Have I kept causality and correlation separate?
- [ ] Didn't I notice skewed or two-peaked distributions and blindly trust the mean?
- [ ] Have I derived at least one modeling aspect/hypothesis from my EDA findings?