Gains:
- Ability to clean raw economic data (missing observations, unit confusion, frequency mismatch) and make it ready for analysis with the help of artificial intelligence
- Ability to print standard economic transformations such as real-nominal conversion, indexation, growth rate, seasonal adjustment as code into artificial intelligence and verify them manually
- Ability to recognize data through exploratory data analysis (summary statistics, distribution, outliers, correlation) and critically read artificial intelligence summaries
Economic data rarely comes ready for analysis. In a table you downloaded from TURKSTAT, some months are missing, one column comes as text with thousand brackets, the exchange rate is in Turkish format like "1.234,56", one series is monthly and the other is quarterly. Most of the economist's time is spent not in analysis, but in making the data ready for analysis. This is where AI is a real, low-risk accelerator: it suggests cleanup steps, writes pandas (Python's data processing library), codifies standard economic transformations. But this unit also has an immutable rule: You read every transformation written by artificial intelligence and verify it manually in at least one observation. If the real-nominal cycle is on the wrong footing, the whole analysis quietly decays.
Cleaning: what problems, how to solve them?
The most common problems in economic data and their logic:
- Incomplete observation. One month/quarter is empty. The solution depends on the context: if it does not actually exist, it is left blank; If there is a technical gap, careful interpolation is done — but the line between fabrication is thin.
- Confusion of units and formats. The text "12.345.6" must be converted to a number; million/billion should become consistent.
- Frequency mismatch. To combine a monthly and quarterly series, one is aggregated (monthly to quarterly) — whether it is average, total or end of period depends on the nature of the indicator (stock/flow distinction).
- Outlier (extreme) values. If an observation deviates wildly: is it a real event (crisis, price hike), or a data error? It is understood before deletion.
- Date alignment. Date formats and period definitions of different series are synchronized.
Attention: Filling in missing data seems innocent, but it is an economic decision. Filling a crisis month with the "average of neighboring months" means deleting that crisis from the analysis. Before filling in, ask "why does this gap exist?" Answer the question.
Standard economic transformations
Transformations and their definitions in the daily repertoire of an economist:
- Nominal → Real. Adjusting for price impact: real value = nominal value / price index × 100. The base year of the deflator (adjusting index) determines the base of the result.
- Indexing. Rescaling a series so that a selected period = 100; It is useful for comparing series of different sizes.
- Growth rate. Annual: (same period this period / last year − 1) × 100. Periodic and annualized rate are different things; Confusing is a common mistake.
- Seasonal correction. Purifying regular seasonal effects (summer tourism, holidays); The raw series and the seasonally adjusted series tell different stories.
- Moving average. Smoothing out the noise and making the trend visible.
- Logarithms and differencing. To examine growth dynamics and stationarity (will deepen in the time series unit).
Tip: With each conversion you will be asked "what happened to the unit and base?" ask. The base of the real series is the base of the deflator; The base of the indexed series is the period you choose. Keeping this written down prevents future mistakes.
Exploratory data analysis (EDA)
It is necessary to recognize the data before entering it into the model. Exploratory data analysis (EDA) includes: summary statistics (mean, median, standard deviation, min-max), shape of the distribution, course over time, outliers and correlation between series. AI quickly generates the code for these steps; Your job is to read the output with an economic eye: "Is this average reasonable? Is this correlation a coincidence or a known relationship?"
Caution: Correlation is only a means of identification here, not proof of causation. The fact that two series move together (e.g., ice cream sales and drownings—both triggered by summer) does not necessarily indicate that one leads to the other. We will deepen this distinction in unit 7.
three mini cases
Case 1 — Incorrect deflator base. An analyst had the artificial intelligence write code to realize the wage series. The code worked, the result looked reasonable — but the analyst manually calculated 2020 in the CALCULATE step and it didn't work. Problem: the artificial intelligence used the deflator with a base of 2003=100 and requested the wage series with 2015 prices; The bases were conflicting. The code was repaired with a single line fix, the entire series fell into place.
Case 2 — Silent volume error. An institution had reduced export data in thousand dollars and imports in million dollars. When the artificial intelligence removed the two for the "foreign trade balance", the result was an absurd deficit. The min-max view in EDA revealed the error: the order of magnitude of the two series differed by a factor of 1000. Once the unit was equalized the balance was correct.
Case 3 — Not deleting the outlier. One month in a series was extraordinarily low. AI suggested "outlier, let's clean it". The analyst investigated: production had actually stopped due to a natural disaster that month. The value was real; Deleting it would distort history. Instead of deletion, it was footnoted. The AI's "clear" recommendation was not followed blindly.
Conversion validation table
Conversion
formula essence
manual checkpoint
realization
nominal / price index × 100
Deflator base = outcome base?
annual growth
(t / t-12month − 1)×100
Calculate a period on paper
indexing
series/base period × 100
Is the base period value 100?
Monthly→quarterly
flow: collect / stock: end of period
Correct collection method?
moving average
last n period average
Is the window length correct?
Four copyable templates
1) Cleaning plan:
I have this raw data: [columns and sample rows]. Come up with a cleaning plan to prepare for analysis: missing value, unit/format issue, date alignment, frequency alignment, possible outliers. Explain in one sentence why each step is necessary. Don't write code yet; let me confirm the plan first.
2) pandas cleanup code:
Write pandas code according to the plan I approved. Let the code indicate what it does at each step with a comment line; It prints the number of rows and missing values after the conversion. Do not silently erase any observation; Report every line you delete.
3) Realization (verifiable):
Realize this nominal series with [price index]. Write clearly the base year of the deflator when you use it; Specify what the base of the resulting series is. Show me the account step by step for a single period so I can verify it manually.
4) Exploratory analysis summary:
Write exploratory analysis code for the following cleaned data: summary statistics, time series plot, outlier scan, and correlation matrix. When interpreting the results, make it clear that the correlations you find are not CAUSAL and list 3 points that stand out to me.
Weak prompt / Strong prompt
Weak prompt:
Clear this data.
AI acts with assumptions without knowing what to clean; You can delete observations, change units, you won't notice.
Powerful prompt:
In this monthly foreign trade data, exports are in "thousand USD" and imports are in "million USD". Make the two equal in "million USD", mark the missing months but DO NOT FILL, align the dates to the end of the month. Print how many rows are affected at each step; silently delete no rows. Then show the balance for a single month in a way that I can manually check.
Common mistakes
- Running the conversion code without reading it. The wrong base/unit silently corrupts the entire analysis.
- Filling in missing data without thinking. Erasing the crisis/disaster period with the average.
- Automatically discard outliers. Mistaking a real event for a data error.
- Confusing seasonal and annual growth. The two are different sizes.
- Combining frequencies incorrectly. Collecting the stock series and processing it as a flow.
- Mistaking the correlation in EDA for causality. Mistaking the recognition tool as evidence.
In summary
Data cleaning and transformation is the invisible but decisive 80 percent of economic analysis. Artificial intelligence dramatically speeds up this mechanical work; but it's up to you to read every cleanup step and every transformation and manually verify at least one observation. Deflator base, unit, frequency and outlier decisions are economic decisions and cannot be blindly left to artificial intelligence. Exploratory analysis introduces data, but correlation there is not causation.
Application task
Download an actual raw series (e.g. monthly CPI and a nominal wage/income series). Create a cleaning plan with the 1st template, have the code generated with the 2nd template, and realize the series with the 3rd template. Then calculate the real value of a single period on paper and compare it with the output of artificial intelligence. If you find a mismatch, diagnose and correct its source (base/unit) and note the progress.
checklist
- [ ] I reviewed and approved the cleanup plan before the code was written.
- [ ] I had every line deleted/changed by the artificial intelligence reported; No silent deletion.
- [ ] When filling in missing data you may ask "why is it empty?" I answered the question.
- [ ] I investigated whether the outlier was a real event or a data error.
- [ ] In the realization, I verified that the deflator base and the outcome base match.
- [ ] I manually recalculated the conversion of at least one period.
- [ ] I did not present correlations in EDA as causation.