Vectorization
In short: Vectorization in NumPy is computing on the whole array at once, with no loop: arithmetic, comparisons, masks with & and |, and choices with np.where.
Problem
A loop for every question
The manager wants the prices with VAT. Right after that they ask how many products cost over €5. With a list every question is a new loop:
It works: €1.20 becomes €1.44, and 8 products are over €5. But for every next question you write one more for, one more empty variable and one more counter. The question itself gets lost between the lines.
The idea
Maths on the whole array
With an array you write the calculation once, for all the numbers:
prices * 1.2 multiplies each of the 25 prices. The loop is inside NumPy, in C. This is called a vectorized operation: it works on the whole array, not on a single number. The knife set at €39 becomes €46.80.
+, -, *, / and ** work this way, and so do functions like np.round(array, 2), which rounds every number to 2 places.
The idea
Array by array
Two arrays of the same length are worked out number by number: the first with the first, the second with the second. In order_items.csv every line has a quantity and a unit price. The value of the line is one times the other:
The first line is 4 units at €7.90, that is €31.60. Without a single loop you have the value of all 152 lines. .sum() adds them up: €1,963.50. We get to summaries like sum in lesson 3.
Try it
What the stock is worth
Put in value the value of the stock for every product: its price times the number in stock.
Solution
import pandas as pd
products = pd.read_csv('products.csv')
prices = products['price'].to_numpy()
stock = products['stock'].to_numpy()
value = prices * stock
valuevalue = prices * stock gives 25 values. The banitsa and the cucumbers are 0: they are sold out. The gift card is €19,980 out of €22,542.30 in total. That is the placeholder 999 from the first lesson. This is why you check every calculation with a look at the data.
The idea
Comparison and mask
A comparison is vectorized too. prices > 5 does not give one True or False, but one for every price:
An array of True and False like this is called a mask. Put in square brackets, it keeps only the places with True. And .sum() of a mask counts the Trues, because in a sum True is 1 and False is 0:
8 prices are over €5: the same answer as with the loop, on one line. The mask from the stock also works on another array of the same length: the names of the sold-out products are the banitsa and the cucumbers.
The idea
Two conditions: &, | and ~
The prices from €2 to €5 are two conditions together. In plain Python you would write and. With arrays and does not work:
and asks “is this whole thing true?” and wants one True or False. The array has 25. NumPy does not guess whether you mean “all” or “at least one”, so it stops with an error.
For masks there are three signs that work number by number:
&is “and”: both conditions are true;|is “or”: at least one is true;~is “not”: it flips the mask.
9 prices are from €2 to €5. Under €1 or over €20 are €0.80, €0.90, €24.90 and €39. The prices of the products that are not in stock are €2.50 and €2.80.
See it
The bracket trap
Every condition around & and | is in brackets. If you leave them out, Python works out & before >= and <=, that is, it starts with 2 & prices:
& between a number and decimal prices makes no sense, so you get a TypeError about bitwise_and. The rule is: every condition in brackets, (prices >= 2) & (prices <= 5).
More precisely, if you are curious
In Python &, | and ~ are bitwise operations on whole numbers. That is why they bind tighter than comparisons: 2 & 6 > 1 is (2 & 6) > 1. NumPy took them for masks because and, or and not cannot be redefined for arrays: they always want one True or False. You will see the same with filters in pandas.
Try it
Cheap and in stock
How many products cost under €3 and are in stock (stock over 0)? Put the count in count.
Solution
import pandas as pd
products = pd.read_csv('products.csv')
prices = products['price'].to_numpy()
stock = products['stock'].to_numpy()
count = ((prices < 3) & (stock > 0)).sum()
countcount = ((prices < 3) & (stock > 0)).sum() gives 12. There are 14 products under €3, but two of them are sold out: the banitsa at €2.50 and the cucumbers at €2.80. With and the code fails with a ValueError, and without the brackets with a TypeError.
The idea
One or the other: np.where
A mask chooses which numbers stay. Sometimes you want all of them, but different depending on the condition. np.where(mask, if_yes, if_no) takes one or the other for every place:
Where the stock is 0 you get “sold out”, otherwise “in stock”. In place of the text you can also have arrays: np.where(prices > 10, prices * 0.9, prices) cuts only the expensive prices by 10%.
Exercise
The price list with VAT
Back to the manager: the prices with VAT and a promotion.
Put in with_vat the prices with VAT (times 1.2), rounded to 2 places. Put in over_10 how many of them are over €10.
Solution
import numpy as np
import pandas as pd
prices = pd.read_csv('products.csv')['price'].to_numpy()
with_vat = np.round(prices * 1.2, 2)
over_10 = (with_vat > 10).sum()
over_10with_vat = np.round(prices * 1.2, 2) and over_10 = (with_vat > 10).sum() gives 7. Without VAT 6 products are over €10. The seventh is the kashkaval cheese: €9.50 with VAT becomes €11.40. If you forget .sum(), what is left is the mask itself, not the count.
Promotion: the products over €10 are 10% cheaper, and the others keep the old price. Put the whole new price list (25 prices) in promo.
Solution
import numpy as np
import pandas as pd
prices = pd.read_csv('products.csv')['price'].to_numpy()
promo = np.where(prices > 10, prices * 0.9, prices)
promopromo = np.where(prices > 10, prices * 0.9, prices). Six prices change: the knife set goes from €39 to €35.10, the coffee beans from €14.90 to €13.41. The other 19 are the same. prices[prices > 10] * 0.9 gives only the six new prices and loses the rest, and prices * 0.9 cuts everything.
Test