Window functions
In short: Window functions with OVER and PARTITION BY calculate over a group of rows without collapsing them: rankings, running totals or the previous row with LAG.
Problem
The average, but also the rows
You can already get the average price of every category with GROUP BY:
But the manager wants a table with all the products, and next to each one the average of its category, to see how far each one is off. GROUP BY has squashed the products: of 25 rows, 7 are left. In the lesson on correlated subqueries you did it with a subquery for every row. There is a shorter way.
The idea
OVER and PARTITION BY
AVG(price) OVER (PARTITION BY category_id) works out the average over the group of the row but does not squash the rows: all 25 products are here, each with the average of its own category.
OVER (…) says which rows to work over. That group of rows is called a window, and a function with OVER is called a window function. PARTITION BY splits the rows into groups, like GROUP BY, but without merging them.
One difference from the correlated subquery: for the gift card the average is 20 €, not NULL. PARTITION BY puts all the NULLs into one group, as GROUP BY does, while p2.category_id = p.category_id was not true for NULL.
The idea
A place in a ranking
ROW_NUMBER() gives the row's number in the window. With ORDER BY in OVER you say how the rows are ordered:
In every category the numbers start again, from the most expensive product: the most expensive item in “Kitchen” (category 8) is the knife set, with number 1.
The natural next step is WHERE place = 1, to keep only the first ones. But:
Window functions are worked out after WHERE: first the rows are picked, then they are arranged into windows. So the ranking goes in a step, and the filter goes outside.
Try it
The most expensive in every category
For every category show its name (column category), its most expensive product and the price (columns name and price). The field already numbers the products. Put it in WITH and take only the first ones.
Solution
WITH ranked AS (
SELECT name, category_id, price,
ROW_NUMBER() OVER (PARTITION BY category_id ORDER BY price DESC) AS place
FROM products
)
SELECT c.name AS category, r.name, r.price
FROM ranked r
JOIN categories c ON c.id = r.category_id
WHERE r.place = 1;WITH ranked AS (…), outside WHERE r.place = 1, and JOIN categories for the name. Six rows, one per subcategory. The gift card has no category, and JOIN leaves it out. This is the pattern for “the top N in every group”: number in a window, filter outside.
See it
Shared places
Now a ranking by items ordered in “Bread and pastry” (category 3), with the three functions side by side:
ROW_NUMBERgives different numbers even when there is a tie. Which of the two is first is decided by the second ordering (here the name). Without it, nothing is promised.RANKgives tied rows the same place and then skips: 1, 1, 3.DENSE_RANKgives the same place and does not skip: 1, 1, 2.
For “the top 3 products”, choose on purpose: with ROW_NUMBER exactly three come out, with RANK everyone who shares a place comes out.
The idea
Running total
With ORDER BY in OVER, the window of every row is “from the start up to here”. How much we have sold from 1 September up to every day (without the cancelled orders):
running_total grows row by row: on 3 October it reaches 415.30 €, and on 19 October it is 480.30 €. This is called a running total.
More precisely, if you are curious
If two rows have the same value in ORDER BY, by default both get the sum including the other one. Here every day is one row, so it cannot happen. If you order orders by date and one day has two orders, add a second ordering, such as ORDER BY ordered_at, id, or say ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW explicitly.
Try it
How many customers we had
For every customer show the name, the sign-up date, and how many customers the shop had on that date, including them (column customers_so_far).
Solution
SELECT name, joined_at,
COUNT(*) OVER (ORDER BY joined_at) AS customers_so_far
FROM customers;COUNT(*) OVER (ORDER BY joined_at): from 1 for Maria to 20 for Yavor. Two customers with the same date would get the same number, but here the dates are all different.
The idea
LAG: the previous row
LAG(column) gives the value from the previous row in the window. How many days pass between Maria's orders?
previous is the date of her previous order. julianday turns a date into a day number, so the difference is the number of days between the two orders. The first order has no previous one and gets NULL. The opposite function is LEAD: the next row.
Exercise
Two tasks
1. The shop window on the website shows two products from every category, the most expensive ones.
For every category show its name (column category) and its two most expensive products with their prices (columns name and price).
Solution
WITH ranked AS (
SELECT name, category_id, price,
ROW_NUMBER() OVER (PARTITION BY category_id ORDER BY price DESC) AS place
FROM products
)
SELECT c.name AS category, r.name, r.price
FROM ranked r
JOIN categories c ON c.id = r.category_id
WHERE r.place <= 2;The same as for the most expensive one, but WHERE r.place <= 2. 12 rows, two from each of the six subcategories.
2. Marketing wants to know the longest time each customer has waited between two of their orders.
For every customer with at least two orders show the name and the longest pause in days between two of their consecutive orders (column longest_pause).
Solution
WITH gaps AS (
SELECT customer_id,
julianday(ordered_at) - julianday(LAG(ordered_at) OVER (PARTITION BY customer_id ORDER BY ordered_at)) AS days
FROM orders
)
SELECT c.name, MAX(g.days) AS longest_pause
FROM gaps g
JOIN customers c ON c.id = g.customer_id
WHERE g.days IS NOT NULL
GROUP BY c.id, c.name;In a step: LAG(ordered_at) OVER (PARTITION BY customer_id ORDER BY ordered_at) and the difference with julianday. Outside, MAX(g.days) per customer, without the NULLs of the first orders. 17 customers. Kalin has only one order, so he is not there. Nikolay waited longest: 314 days.
Test
Check yourself
Related lessons
Section titled "Related lessons"- Builds on: JOIN: two tables in one query · Subqueries · Correlated subqueries · A report in steps: WITH
- Continues in: Event streams · Columnar and time series databases · The bakery, stage 6: the report, the copy and DROP
- Tasks on this topic: Subqueries and windows