Welcome

In short: Data analysis is the path from the question to the answer: loading, cleaning, computing, charts and checking, here with Python, pandas and matplotlib right in the browser.

Problem

The number is right. The decision is wrong

The shop sees that revenue in December fell tenfold compared with October. The manager stops the ads, changes the supplier and cuts prices. A month later it turns out that in October two customers placed big orders for their office. December was an ordinary month.

The number in the table was right. What was wrong was what followed from it. Data analysis is exactly this: answering a question with data so that the answer can be trusted.

The idea

The path of an analysis

Every analysis goes through the same steps, and the course follows them:

1. The question. What does someone have to decide with the answer? “Why did revenue fall” is a question. “Analyse the sales” is not.
2. The data. Which tables, from when, what one row means.
3. Cleaning. Empty cells, duplicates, “1,60 €” as text, wrong dates.
4. Exploring. Tables, groups and charts: what is in the data.
5. The conclusion. Is the drop real, or noise? A cause, or a coincidence?
6. The story. Three sentences, one chart and the limits of the answer.

Here is where you are going: the shop’s revenue by month, from the raw tables to the chart. Press “Run”:

Pythondata: the shop tables

import pandas as pd

orders = pd.read_csv('orders.csv')
items = pd.read_csv('order_items.csv')
lines = items.merge(orders, left_on='order_id', right_on='id')
lines = lines[lines['status'] != 'cancelled']
lines['revenue'] = lines['quantity'] * lines['unit_price']
lines['month'] = lines['ordered_at'].str[:7]
monthly = lines.groupby('month')['revenue'].sum()
monthly.plot.bar(title='Revenue by month, €')

import pandas as pd orders = pd.read_csv('orders.csv') items = pd.read_csv('order_items.csv') lines = items.merge(orders, left_on='order_id', right_on='id') lines = lines[lines['status'] != 'cancelled'] lines['revenue'] = lines['quantity'] * lines['unit_price'] lines['month'] = lines['ordered_at'].str[:7] monthly = lines.groupby('month')['revenue'].sum() monthly.plot.bar(title='Revenue by month, €')

▶ RunRestore the code

The code will be clear line by line in the “Wrangling data” module. The chart, though, already asks a question: the months jump a lot. Is December a drop, or noise? The answer is in the statistics module.

The idea

The sandbox

The code in the lessons is real Python, and it runs in your browser. The first run downloads Python (about 13 MB), and afterwards it is in the cache. Try it, then change the prices and run it again:

Pythondata: the shop tables

prices = [1.2, 1.9, 0.8, 3.2]
print('Basket:', round(sum(prices), 2), '€')
len(prices)

prices = [1.2, 1.9, 0.8, 3.2] print('Basket:', round(sum(prices), 2), '€') len(prices)

▶ RunRestore the code

• Ctrl+Enter (⌘+Enter on a Mac) runs the code, and “Restore the code” brings the example back. Tab indents.
• The output is like a notebook’s: everything from print, and at the end the value of the last line. Tables come out as tables, and charts as pictures.
• An error is explained: first in words and with the line number, and under it is Python’s original, so you recognise it when you see it on your own computer.
• Every run starts afresh. The variables from the previous run are gone, as if you ran the whole notebook from the top.
• There is no network. The data is the files next to the code: the shop tables from the SQL course, in euros.
• Code that does not finish in 10 seconds (an endless loop) is stopped, and the page does not freeze.

The idea

The modules

• NumPy: arrays, and maths on thousands of numbers without for.
• Tables with pandas: selecting, filtering, new columns, types and missing values.
• Wrangling data: groups, joining tables, wide and long, dates, dirty data.
• Charts: which chart for which question, and how a chart lies.
• Statistics for decisions: how noisy a number is, intervals, hypothesis tests, an A/B test, correlation and cause.
• Models with scikit-learn: prediction, classification, leakage and an honest check.
• Project: why revenue fell: the whole path with the shop’s data.

More precisely, if you are curious

Data analysis is also done in R (strong in statistics) and in Julia (fast for numerical work). The industry is mostly on Python, so the course is in it. The ideas are the same.

The idea

What you need to know

From Python you need lists, dicts, for, functions and lambdas: the Python course up to “Collections”. SQL helps, but you do not need it: grouping and joining are taught again here, with pandas.

In the browser you need nothing. For your own work you need Python and JupyterLab: the notebook, where code is in cells and the output is under each one. With uv you make the project like this:

uv init shop-analysis
cd shop-analysis
uv add pandas matplotlib scikit-learn jupyterlab
uv run jupyter lab

Without uv, a plain virtual environment works too:

python -m venv .venv
source .venv/bin/activate
pip install pandas matplotlib scikit-learn jupyterlab
jupyter lab

On Windows the second command is .venv\Scripts\activate. VS Code also opens notebooks (the .ipynb files) with the Jupyter extension. These commands do not run here.

Test

Check yourself

1. What comes first in data analysis?

The chartThe question: what someone has to decide with the answerThe machine learning model

2. The code in the sandbox calls pd.read_csv("https://example.com/sales.csv"). What happens?

The page reloadsAn error: the sandbox has no network, the data is the files next to the codeThe file is downloaded from the internet

3. Revenue in December is €22.70, and in October it was €255. What is the honest conclusion?

The shop is losing customersNone yet: with 60 orders a year one month may simply be noisePeople do not shop in December

Check

Start the lesson over