Classification

In short: Classification predicts a class, such as “will buy again” or not; logistic regression gives a probability, and the threshold trades precision for recall depending on which mistakes cost more.

Problem

“Will the customer buy again?”

Marketing wants to send a discount coupon to the customers who would otherwise stop buying. A coupon for everyone costs a lot, and most customers come back anyway. The question is: to whom?

Let’s go back in time. Stand on 1 July 2026: this is the cutoff. For each customer you know their orders up to that date. How many, how many days ago the last one was, how big the average basket is. Whether they order again you learn from July to the end of October:

Pythondata: the shop tables

import pandas as pd

orders = pd.read_csv('year_orders.csv', parse_dates=['ordered_at'])
items = pd.read_csv('year_items.csv')
items['value'] = items['quantity'] * items['unit_price']
orders['basket'] = orders['id'].map(items.groupby('order_id')['value'].sum())

cutoff = pd.Timestamp('2026-07-01')
before = orders[orders['ordered_at'] < cutoff]
after = orders[orders['ordered_at'] >= cutoff]

people = before.groupby('customer_id').agg(
    orders=('id', 'size'), last=('ordered_at', 'max'), basket=('basket', 'mean'))
people['days_since'] = (cutoff - people['last']).dt.days
people['lost'] = (~people.index.isin(after['customer_id'])).astype(int)

print('customers before 1 July:', len(people))
print('came back after it:', round(1 - people['lost'].mean(), 3))
people[['orders', 'days_since', 'basket', 'lost']].round(2).sort_values('orders').head()

import pandas as pd orders = pd.read_csv('year_orders.csv', parse_dates=['ordered_at']) items = pd.read_csv('year_items.csv') items['value'] = items['quantity'] * items['unit_price'] orders['basket'] = orders['id'].map(items.groupby('order_id')['value'].sum()) cutoff = pd.Timestamp('2026-07-01') before = orders[orders['ordered_at'] < cutoff] after = orders[orders['ordered_at'] >= cutoff] people = before.groupby('customer_id').agg( orders=('id', 'size'), last=('ordered_at', 'max'), basket=('basket', 'mean')) people['days_since'] = (cutoff - people['last']).dt.days people['lost'] = (~people.index.isin(after['customer_id'])).astype(int) print('customers before 1 July:', len(people)) print('came back after it:', round(1 - people['lost'].mean(), 3)) people[['orders', 'days_since', 'basket', 'lost']].round(2).sort_values('orders').head()

▶ RunRestore the code

Before 1 July, 1,279 customers bought something. After it, 72.6% of them ordered again. The lost column is 1 for the rest: they ordered nothing for 4 months. These are the people the coupon is for.

More precisely, if you are curious

Here we count every order, the cancelled ones too: the question is whether the customer comes, not how much revenue they bring. basket is the average value of their basket. ~people.index.isin(...) is the mask “the customer is missing after the cutoff”: ~ flips True and False.

Try it

How much comes free?

Store in lost_share what share of the customers is lost, and in baseline_acc how often the rule “everyone comes back” is right, without any model.

Pythondata: the shop tables

import pandas as pd

orders = pd.read_csv('year_orders.csv', parse_dates=['ordered_at'])
items = pd.read_csv('year_items.csv')
items['value'] = items['quantity'] * items['unit_price']
orders['basket'] = orders['id'].map(items.groupby('order_id')['value'].sum())

cutoff = pd.Timestamp('2026-07-01')
before = orders[orders['ordered_at'] < cutoff]
after = orders[orders['ordered_at'] >= cutoff]

people = before.groupby('customer_id').agg(
    orders=('id', 'size'), last=('ordered_at', 'max'), basket=('basket', 'mean'))
people['days_since'] = (cutoff - people['last']).dt.days
people['lost'] = (~people.index.isin(after['customer_id'])).astype(int)

lost_share = ...
baseline_acc = ...
print(lost_share, baseline_acc)

import pandas as pd orders = pd.read_csv('year_orders.csv', parse_dates=['ordered_at']) items = pd.read_csv('year_items.csv') items['value'] = items['quantity'] * items['unit_price'] orders['basket'] = orders['id'].map(items.groupby('order_id')['value'].sum()) cutoff = pd.Timestamp('2026-07-01') before = orders[orders['ordered_at'] < cutoff] after = orders[orders['ordered_at'] >= cutoff] people = before.groupby('customer_id').agg( orders=('id', 'size'), last=('ordered_at', 'max'), basket=('basket', 'mean')) people['days_since'] = (cutoff - people['last']).dt.days people['lost'] = (~people.index.isin(after['customer_id'])).astype(int) lost_share = ... baseline_acc = ... print(lost_share, baseline_acc)

Check▶ Run

The idea

Classification: the answer is a class

In regression the prediction is a number: revenue for a day. Here the answer is “yes” or “no”: is the customer lost. Such a task is classification, and the possible answers are classes. The path is the same: features X, label y, split, fit, predict.

The first model is logistic regression (LogisticRegression). The name is misleading: it classifies. Like linear regression, it adds up the features with weights, but then it squeezes the sum into a chance between 0 and 1. Against it we put DummyClassifier, which always says the most common class:

Pythondata: the shop tables

import pandas as pd

orders = pd.read_csv('year_orders.csv', parse_dates=['ordered_at'])
items = pd.read_csv('year_items.csv')
items['value'] = items['quantity'] * items['unit_price']
orders['basket'] = orders['id'].map(items.groupby('order_id')['value'].sum())

cutoff = pd.Timestamp('2026-07-01')
before = orders[orders['ordered_at'] < cutoff]
after = orders[orders['ordered_at'] >= cutoff]

people = before.groupby('customer_id').agg(
    orders=('id', 'size'), last=('ordered_at', 'max'), basket=('basket', 'mean'))
people['days_since'] = (cutoff - people['last']).dt.days
people['lost'] = (~people.index.isin(after['customer_id'])).astype(int)

from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.dummy import DummyClassifier

X = people[['orders', 'days_since', 'basket']]
y = people['lost']
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, random_state=42, stratify=y)

baseline = DummyClassifier(strategy='most_frequent').fit(X_train, y_train)
model = LogisticRegression(max_iter=1000).fit(X_train, y_train)
print('customers for training and for test:', len(X_train), len(X_test))
print('accuracy, “everyone comes back”:', round(baseline.score(X_test, y_test), 3))
print('accuracy, the model:', round(model.score(X_test, y_test), 3))

import pandas as pd orders = pd.read_csv('year_orders.csv', parse_dates=['ordered_at']) items = pd.read_csv('year_items.csv') items['value'] = items['quantity'] * items['unit_price'] orders['basket'] = orders['id'].map(items.groupby('order_id')['value'].sum()) cutoff = pd.Timestamp('2026-07-01') before = orders[orders['ordered_at'] < cutoff] after = orders[orders['ordered_at'] >= cutoff] people = before.groupby('customer_id').agg( orders=('id', 'size'), last=('ordered_at', 'max'), basket=('basket', 'mean')) people['days_since'] = (cutoff - people['last']).dt.days people['lost'] = (~people.index.isin(after['customer_id'])).astype(int) from sklearn.model_selection import train_test_split from sklearn.linear_model import LogisticRegression from sklearn.dummy import DummyClassifier X = people[['orders', 'days_since', 'basket']] y = people['lost'] X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.25, random_state=42, stratify=y) baseline = DummyClassifier(strategy='most_frequent').fit(X_train, y_train) model = LogisticRegression(max_iter=1000).fit(X_train, y_train) print('customers for training and for test:', len(X_train), len(X_test)) print('accuracy, “everyone comes back”:', round(baseline.score(X_test, y_test), 3)) print('accuracy, the model:', round(model.score(X_test, y_test), 3))

▶ RunRestore the code

The rule “everyone comes back” is right about 72.5% of the test customers. The model is right about 72.2%: less than the rule. Accuracy (the share of right answers) lies here. When about one customer in four is lost, “everyone stays” is almost always true.

More precisely, if you are curious

stratify=y splits so that the share of lost customers is the same in training and in the test: 27.5%. Without it, with unequal classes, the test may by chance get more or fewer lost customers. Here it is fair to split customers at random: each customer’s features are from before the cutoff, and the label is from after it. max_iter=1000 gives the model more steps to find the weights: the features are on different scales (orders up to 77, days up to 240).

The idea

The confusion matrix

Accuracy is one number for all errors. But the errors are not the same. A missed lost customer is lost income. A coupon for a customer who would have come back is a discount given away. The table that counts them separately is called the confusion matrix:

Pythondata: the shop tables

import pandas as pd

orders = pd.read_csv('year_orders.csv', parse_dates=['ordered_at'])
items = pd.read_csv('year_items.csv')
items['value'] = items['quantity'] * items['unit_price']
orders['basket'] = orders['id'].map(items.groupby('order_id')['value'].sum())

cutoff = pd.Timestamp('2026-07-01')
before = orders[orders['ordered_at'] < cutoff]
after = orders[orders['ordered_at'] >= cutoff]

people = before.groupby('customer_id').agg(
    orders=('id', 'size'), last=('ordered_at', 'max'), basket=('basket', 'mean'))
people['days_since'] = (cutoff - people['last']).dt.days
people['lost'] = (~people.index.isin(after['customer_id'])).astype(int)

from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression

X = people[['orders', 'days_since', 'basket']]
y = people['lost']
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, random_state=42, stratify=y)
model = LogisticRegression(max_iter=1000).fit(X_train, y_train)
from sklearn.metrics import confusion_matrix

pred = model.predict(X_test)
table = confusion_matrix(y_test, pred)
pd.DataFrame(table, index=['really stays', 'really lost'],
             columns=['model: stays', 'model: lost'])

import pandas as pd orders = pd.read_csv('year_orders.csv', parse_dates=['ordered_at']) items = pd.read_csv('year_items.csv') items['value'] = items['quantity'] * items['unit_price'] orders['basket'] = orders['id'].map(items.groupby('order_id')['value'].sum()) cutoff = pd.Timestamp('2026-07-01') before = orders[orders['ordered_at'] < cutoff] after = orders[orders['ordered_at'] >= cutoff] people = before.groupby('customer_id').agg( orders=('id', 'size'), last=('ordered_at', 'max'), basket=('basket', 'mean')) people['days_since'] = (cutoff - people['last']).dt.days people['lost'] = (~people.index.isin(after['customer_id'])).astype(int) from sklearn.model_selection import train_test_split from sklearn.linear_model import LogisticRegression X = people[['orders', 'days_since', 'basket']] y = people['lost'] X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.25, random_state=42, stratify=y) model = LogisticRegression(max_iter=1000).fit(X_train, y_train) from sklearn.metrics import confusion_matrix pred = model.predict(X_test) table = confusion_matrix(y_test, pred) pd.DataFrame(table, index=['really stays', 'really lost'], columns=['model: stays', 'model: lost'])

▶ RunRestore the code

The rows are the truth, the columns are what the model says. Of 88 lost customers the model points to 1. It misses the other 87. That is why it is as accurate as the rule: it too almost always says “stays”.

The idea

Precision and recall: who gets a coupon

Two numbers describe the coupons better than accuracy:

• precision: of the customers with a coupon, how many really would have left. Low means many discounts given away;
• recall: of everyone who would have left, how many get a coupon. Low means many missed customers.

Pythondata: the shop tables

import pandas as pd

orders = pd.read_csv('year_orders.csv', parse_dates=['ordered_at'])
items = pd.read_csv('year_items.csv')
items['value'] = items['quantity'] * items['unit_price']
orders['basket'] = orders['id'].map(items.groupby('order_id')['value'].sum())

cutoff = pd.Timestamp('2026-07-01')
before = orders[orders['ordered_at'] < cutoff]
after = orders[orders['ordered_at'] >= cutoff]

people = before.groupby('customer_id').agg(
    orders=('id', 'size'), last=('ordered_at', 'max'), basket=('basket', 'mean'))
people['days_since'] = (cutoff - people['last']).dt.days
people['lost'] = (~people.index.isin(after['customer_id'])).astype(int)

from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression

X = people[['orders', 'days_since', 'basket']]
y = people['lost']
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, random_state=42, stratify=y)
model = LogisticRegression(max_iter=1000).fit(X_train, y_train)
from sklearn.metrics import precision_score, recall_score

pred = model.predict(X_test)
print('coupons:', pred.sum())
print('precision:', round(precision_score(y_test, pred), 3))
print('recall:', round(recall_score(y_test, pred), 3))

import pandas as pd orders = pd.read_csv('year_orders.csv', parse_dates=['ordered_at']) items = pd.read_csv('year_items.csv') items['value'] = items['quantity'] * items['unit_price'] orders['basket'] = orders['id'].map(items.groupby('order_id')['value'].sum()) cutoff = pd.Timestamp('2026-07-01') before = orders[orders['ordered_at'] < cutoff] after = orders[orders['ordered_at'] >= cutoff] people = before.groupby('customer_id').agg( orders=('id', 'size'), last=('ordered_at', 'max'), basket=('basket', 'mean')) people['days_since'] = (cutoff - people['last']).dt.days people['lost'] = (~people.index.isin(after['customer_id'])).astype(int) from sklearn.model_selection import train_test_split from sklearn.linear_model import LogisticRegression X = people[['orders', 'days_since', 'basket']] y = people['lost'] X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.25, random_state=42, stratify=y) model = LogisticRegression(max_iter=1000).fit(X_train, y_train) from sklearn.metrics import precision_score, recall_score pred = model.predict(X_test) print('coupons:', pred.sum()) print('precision:', round(precision_score(y_test, pred), 3)) print('recall:', round(recall_score(y_test, pred), 3))

▶ RunRestore the code

The model gives 3 coupons. One goes to a lost customer: precision is 0.333. Of the 88 lost it catches 1: recall is 0.011. Such a model does not help marketing, whatever its accuracy.

The idea

The chance and the threshold

predict is not all the model knows. predict_proba gives a chance for each class. The columns follow the order of model.classes_: the first is for 0 (stays), the second for 1 (lost):

Pythondata: the shop tables

import pandas as pd

orders = pd.read_csv('year_orders.csv', parse_dates=['ordered_at'])
items = pd.read_csv('year_items.csv')
items['value'] = items['quantity'] * items['unit_price']
orders['basket'] = orders['id'].map(items.groupby('order_id')['value'].sum())

cutoff = pd.Timestamp('2026-07-01')
before = orders[orders['ordered_at'] < cutoff]
after = orders[orders['ordered_at'] >= cutoff]

people = before.groupby('customer_id').agg(
    orders=('id', 'size'), last=('ordered_at', 'max'), basket=('basket', 'mean'))
people['days_since'] = (cutoff - people['last']).dt.days
people['lost'] = (~people.index.isin(after['customer_id'])).astype(int)

from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression

X = people[['orders', 'days_since', 'basket']]
y = people['lost']
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, random_state=42, stratify=y)
model = LogisticRegression(max_iter=1000).fit(X_train, y_train)
proba = model.predict_proba(X_test)[:, 1]

print(model.classes_)
print(model.predict_proba(X_test)[:3].round(2))
print('the highest chance of loss:', proba.max().round(2))
print('at least 0.5:', (proba >= 0.5).sum(), ' at least 0.3:', (proba >= 0.3).sum())

import pandas as pd orders = pd.read_csv('year_orders.csv', parse_dates=['ordered_at']) items = pd.read_csv('year_items.csv') items['value'] = items['quantity'] * items['unit_price'] orders['basket'] = orders['id'].map(items.groupby('order_id')['value'].sum()) cutoff = pd.Timestamp('2026-07-01') before = orders[orders['ordered_at'] < cutoff] after = orders[orders['ordered_at'] >= cutoff] people = before.groupby('customer_id').agg( orders=('id', 'size'), last=('ordered_at', 'max'), basket=('basket', 'mean')) people['days_since'] = (cutoff - people['last']).dt.days people['lost'] = (~people.index.isin(after['customer_id'])).astype(int) from sklearn.model_selection import train_test_split from sklearn.linear_model import LogisticRegression X = people[['orders', 'days_since', 'basket']] y = people['lost'] X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.25, random_state=42, stratify=y) model = LogisticRegression(max_iter=1000).fit(X_train, y_train) proba = model.predict_proba(X_test)[:, 1] print(model.classes_) print(model.predict_proba(X_test)[:3].round(2)) print('the highest chance of loss:', proba.max().round(2)) print('at least 0.5:', (proba >= 0.5).sum(), ' at least 0.3:', (proba >= 0.3).sum())

▶ RunRestore the code

The highest chance of loss in the test is 0.56. predict gives a coupon only if the chance is over 0.5, and there are 3 such customers. But 0.5 is not a law. The number above which you say “yes” is called the threshold, and you choose it. At a threshold of 0.3 there are 160 coupons:

Pythondata: the shop tables

import pandas as pd

orders = pd.read_csv('year_orders.csv', parse_dates=['ordered_at'])
items = pd.read_csv('year_items.csv')
items['value'] = items['quantity'] * items['unit_price']
orders['basket'] = orders['id'].map(items.groupby('order_id')['value'].sum())

cutoff = pd.Timestamp('2026-07-01')
before = orders[orders['ordered_at'] < cutoff]
after = orders[orders['ordered_at'] >= cutoff]

people = before.groupby('customer_id').agg(
    orders=('id', 'size'), last=('ordered_at', 'max'), basket=('basket', 'mean'))
people['days_since'] = (cutoff - people['last']).dt.days
people['lost'] = (~people.index.isin(after['customer_id'])).astype(int)

from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression

X = people[['orders', 'days_since', 'basket']]
y = people['lost']
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, random_state=42, stratify=y)
model = LogisticRegression(max_iter=1000).fit(X_train, y_train)
proba = model.predict_proba(X_test)[:, 1]
from sklearn.metrics import confusion_matrix, precision_score, recall_score

pred = (proba >= 0.3).astype(int)
print(confusion_matrix(y_test, pred))
print('coupons:', pred.sum())
print('precision:', round(precision_score(y_test, pred), 3))
print('recall:', round(recall_score(y_test, pred), 3))

import pandas as pd orders = pd.read_csv('year_orders.csv', parse_dates=['ordered_at']) items = pd.read_csv('year_items.csv') items['value'] = items['quantity'] * items['unit_price'] orders['basket'] = orders['id'].map(items.groupby('order_id')['value'].sum()) cutoff = pd.Timestamp('2026-07-01') before = orders[orders['ordered_at'] < cutoff] after = orders[orders['ordered_at'] >= cutoff] people = before.groupby('customer_id').agg( orders=('id', 'size'), last=('ordered_at', 'max'), basket=('basket', 'mean')) people['days_since'] = (cutoff - people['last']).dt.days people['lost'] = (~people.index.isin(after['customer_id'])).astype(int) from sklearn.model_selection import train_test_split from sklearn.linear_model import LogisticRegression X = people[['orders', 'days_since', 'basket']] y = people['lost'] X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.25, random_state=42, stratify=y) model = LogisticRegression(max_iter=1000).fit(X_train, y_train) proba = model.predict_proba(X_test)[:, 1] from sklearn.metrics import confusion_matrix, precision_score, recall_score pred = (proba >= 0.3).astype(int) print(confusion_matrix(y_test, pred)) print('coupons:', pred.sum()) print('precision:', round(precision_score(y_test, pred), 3)) print('recall:', round(recall_score(y_test, pred), 3))

▶ RunRestore the code

Of 160 coupons, 63 go to lost customers: precision is 39.4%. Of the 88 lost you catch 63: recall is 71.6%. Accuracy drops, but now the model works. For comparison: 160 coupons to random customers would have a precision equal to the share of lost ones, 27.5%. The model ranks customers better than chance, though far from perfectly.

More precisely, if you are curious

Another model gives a similar picture. A decision tree with depth 3 is also exactly as accurate as the rule, and at a threshold of 0.3 it has a precision of 38.8% and a recall of 64.8%:

Pythondata: the shop tables

import pandas as pd

orders = pd.read_csv('year_orders.csv', parse_dates=['ordered_at'])
items = pd.read_csv('year_items.csv')
items['value'] = items['quantity'] * items['unit_price']
orders['basket'] = orders['id'].map(items.groupby('order_id')['value'].sum())

cutoff = pd.Timestamp('2026-07-01')
before = orders[orders['ordered_at'] < cutoff]
after = orders[orders['ordered_at'] >= cutoff]

people = before.groupby('customer_id').agg(
    orders=('id', 'size'), last=('ordered_at', 'max'), basket=('basket', 'mean'))
people['days_since'] = (cutoff - people['last']).dt.days
people['lost'] = (~people.index.isin(after['customer_id'])).astype(int)

from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression

X = people[['orders', 'days_since', 'basket']]
y = people['lost']
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, random_state=42, stratify=y)
model = LogisticRegression(max_iter=1000).fit(X_train, y_train)
from sklearn.tree import DecisionTreeClassifier
from sklearn.metrics import recall_score, precision_score

tree = DecisionTreeClassifier(max_depth=3, random_state=42).fit(X_train, y_train)
print('accuracy:', round(tree.score(X_test, y_test), 3))
pred = (tree.predict_proba(X_test)[:, 1] >= 0.3).astype(int)
print('at threshold 0.3: precision', round(precision_score(y_test, pred), 3), 'recall', round(recall_score(y_test, pred), 3))

import pandas as pd orders = pd.read_csv('year_orders.csv', parse_dates=['ordered_at']) items = pd.read_csv('year_items.csv') items['value'] = items['quantity'] * items['unit_price'] orders['basket'] = orders['id'].map(items.groupby('order_id')['value'].sum()) cutoff = pd.Timestamp('2026-07-01') before = orders[orders['ordered_at'] < cutoff] after = orders[orders['ordered_at'] >= cutoff] people = before.groupby('customer_id').agg( orders=('id', 'size'), last=('ordered_at', 'max'), basket=('basket', 'mean')) people['days_since'] = (cutoff - people['last']).dt.days people['lost'] = (~people.index.isin(after['customer_id'])).astype(int) from sklearn.model_selection import train_test_split from sklearn.linear_model import LogisticRegression X = people[['orders', 'days_since', 'basket']] y = people['lost'] X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.25, random_state=42, stratify=y) model = LogisticRegression(max_iter=1000).fit(X_train, y_train) from sklearn.tree import DecisionTreeClassifier from sklearn.metrics import recall_score, precision_score tree = DecisionTreeClassifier(max_depth=3, random_state=42).fit(X_train, y_train) print('accuracy:', round(tree.score(X_test, y_test), 3)) pred = (tree.predict_proba(X_test)[:, 1] >= 0.3).astype(int) print('at threshold 0.3: precision', round(precision_score(y_test, pred), 3), 'recall', round(recall_score(y_test, pred), 3))

▶ RunRestore the code

With these three features you cannot get much more. If you want a better model, look for features that say more about the customer: what products they buy, how regularly they order.

Try it

Count them yourself

At a threshold of 0.4, count fp (a coupon for a customer who stays) and fn (a lost customer without a coupon), then compute precision and recall from tp, fp and fn, without sklearn.

Pythondata: the shop tables

import pandas as pd

orders = pd.read_csv('year_orders.csv', parse_dates=['ordered_at'])
items = pd.read_csv('year_items.csv')
items['value'] = items['quantity'] * items['unit_price']
orders['basket'] = orders['id'].map(items.groupby('order_id')['value'].sum())

cutoff = pd.Timestamp('2026-07-01')
before = orders[orders['ordered_at'] < cutoff]
after = orders[orders['ordered_at'] >= cutoff]

people = before.groupby('customer_id').agg(
    orders=('id', 'size'), last=('ordered_at', 'max'), basket=('basket', 'mean'))
people['days_since'] = (cutoff - people['last']).dt.days
people['lost'] = (~people.index.isin(after['customer_id'])).astype(int)

from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression

X = people[['orders', 'days_since', 'basket']]
y = people['lost']
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, random_state=42, stratify=y)
model = LogisticRegression(max_iter=1000).fit(X_train, y_train)
proba = model.predict_proba(X_test)[:, 1]

pred = (proba >= 0.4).astype(int)
truth = y_test.to_numpy()
tp = ((pred == 1) & (truth == 1)).sum()
fp = ...
fn = ...
precision = ...
recall = ...
print(tp, fp, fn, precision, recall)

import pandas as pd orders = pd.read_csv('year_orders.csv', parse_dates=['ordered_at']) items = pd.read_csv('year_items.csv') items['value'] = items['quantity'] * items['unit_price'] orders['basket'] = orders['id'].map(items.groupby('order_id')['value'].sum()) cutoff = pd.Timestamp('2026-07-01') before = orders[orders['ordered_at'] < cutoff] after = orders[orders['ordered_at'] >= cutoff] people = before.groupby('customer_id').agg( orders=('id', 'size'), last=('ordered_at', 'max'), basket=('basket', 'mean')) people['days_since'] = (cutoff - people['last']).dt.days people['lost'] = (~people.index.isin(after['customer_id'])).astype(int) from sklearn.model_selection import train_test_split from sklearn.linear_model import LogisticRegression X = people[['orders', 'days_since', 'basket']] y = people['lost'] X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.25, random_state=42, stratify=y) model = LogisticRegression(max_iter=1000).fit(X_train, y_train) proba = model.predict_proba(X_test)[:, 1] pred = (proba >= 0.4).astype(int) truth = y_test.to_numpy() tp = ((pred == 1) & (truth == 1)).sum() fp = ... fn = ... precision = ... recall = ... print(tp, fp, fn, precision, recall)

Check▶ Run

The idea

The threshold trades one for the other

Each threshold gives different coupons. Here are all thresholds from 0.1 to 0.5:

Pythondata: the shop tables

import pandas as pd

orders = pd.read_csv('year_orders.csv', parse_dates=['ordered_at'])
items = pd.read_csv('year_items.csv')
items['value'] = items['quantity'] * items['unit_price']
orders['basket'] = orders['id'].map(items.groupby('order_id')['value'].sum())

cutoff = pd.Timestamp('2026-07-01')
before = orders[orders['ordered_at'] < cutoff]
after = orders[orders['ordered_at'] >= cutoff]

people = before.groupby('customer_id').agg(
    orders=('id', 'size'), last=('ordered_at', 'max'), basket=('basket', 'mean'))
people['days_since'] = (cutoff - people['last']).dt.days
people['lost'] = (~people.index.isin(after['customer_id'])).astype(int)

from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression

X = people[['orders', 'days_since', 'basket']]
y = people['lost']
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, random_state=42, stratify=y)
model = LogisticRegression(max_iter=1000).fit(X_train, y_train)
proba = model.predict_proba(X_test)[:, 1]
import numpy as np
import matplotlib.pyplot as plt
from sklearn.metrics import precision_score, recall_score

rows = []
for t in np.arange(0.10, 0.51, 0.05).round(2):
    pred = (proba >= t).astype(int)
    rows.append({'threshold': t, 'coupons': pred.sum(),
                 'precision': precision_score(y_test, pred, zero_division=0),
                 'recall': recall_score(y_test, pred)})
curve = pd.DataFrame(rows).set_index('threshold').round(3)
print(curve)

fig, ax = plt.subplots(figsize=(7, 3.5))
ax.plot(curve.index, curve['precision'], marker='o', label='precision')
ax.plot(curve.index, curve['recall'], marker='o', label='recall')
ax.set_title('The threshold: more coupons or more precise coupons')
ax.set_xlabel('threshold on the chance of loss')
ax.set_ylim(0, 1)
ax.legend()
fig.tight_layout()

import pandas as pd orders = pd.read_csv('year_orders.csv', parse_dates=['ordered_at']) items = pd.read_csv('year_items.csv') items['value'] = items['quantity'] * items['unit_price'] orders['basket'] = orders['id'].map(items.groupby('order_id')['value'].sum()) cutoff = pd.Timestamp('2026-07-01') before = orders[orders['ordered_at'] < cutoff] after = orders[orders['ordered_at'] >= cutoff] people = before.groupby('customer_id').agg( orders=('id', 'size'), last=('ordered_at', 'max'), basket=('basket', 'mean')) people['days_since'] = (cutoff - people['last']).dt.days people['lost'] = (~people.index.isin(after['customer_id'])).astype(int) from sklearn.model_selection import train_test_split from sklearn.linear_model import LogisticRegression X = people[['orders', 'days_since', 'basket']] y = people['lost'] X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.25, random_state=42, stratify=y) model = LogisticRegression(max_iter=1000).fit(X_train, y_train) proba = model.predict_proba(X_test)[:, 1] import numpy as np import matplotlib.pyplot as plt from sklearn.metrics import precision_score, recall_score rows = [] for t in np.arange(0.10, 0.51, 0.05).round(2): pred = (proba >= t).astype(int) rows.append({'threshold': t, 'coupons': pred.sum(), 'precision': precision_score(y_test, pred, zero_division=0), 'recall': recall_score(y_test, pred)}) curve = pd.DataFrame(rows).set_index('threshold').round(3) print(curve) fig, ax = plt.subplots(figsize=(7, 3.5)) ax.plot(curve.index, curve['precision'], marker='o', label='precision') ax.plot(curve.index, curve['recall'], marker='o', label='recall') ax.set_title('The threshold: more coupons or more precise coupons') ax.set_xlabel('threshold on the chance of loss') ax.set_ylim(0, 1) ax.legend() fig.tight_layout()

▶ RunRestore the code

At a threshold of 0.1 there are 262 coupons and you catch 96.6% of the lost customers, but precision is 32.4%: a little over the 27.5% of random coupons. At 0.4 there are 81 coupons and precision is 45.7%, but you miss more than half of the lost customers. At 0.5 there are only 3 coupons. No threshold is best on its own. The choice depends on the cost: what a coupon costs and what a lost customer costs.

Exercise

Marketing’s coupons

Marketing has 80 coupons for the customers in the test. Give them to the 80 with the highest chance of loss: proba.argsort() gives the positions from the smallest to the largest chance. Store in hits how many of them are really lost, in precision their share among the 80, and in recall what share of all lost customers in the test gets a coupon.

Pythondata: the shop tables

import pandas as pd

orders = pd.read_csv('year_orders.csv', parse_dates=['ordered_at'])
items = pd.read_csv('year_items.csv')
items['value'] = items['quantity'] * items['unit_price']
orders['basket'] = orders['id'].map(items.groupby('order_id')['value'].sum())

cutoff = pd.Timestamp('2026-07-01')
before = orders[orders['ordered_at'] < cutoff]
after = orders[orders['ordered_at'] >= cutoff]

people = before.groupby('customer_id').agg(
    orders=('id', 'size'), last=('ordered_at', 'max'), basket=('basket', 'mean'))
people['days_since'] = (cutoff - people['last']).dt.days
people['lost'] = (~people.index.isin(after['customer_id'])).astype(int)

from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression

X = people[['orders', 'days_since', 'basket']]
y = people['lost']
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, random_state=42, stratify=y)
model = LogisticRegression(max_iter=1000).fit(X_train, y_train)
proba = model.predict_proba(X_test)[:, 1]

top = ...
hits = ...
precision = ...
recall = ...
print(hits, precision, recall)

import pandas as pd orders = pd.read_csv('year_orders.csv', parse_dates=['ordered_at']) items = pd.read_csv('year_items.csv') items['value'] = items['quantity'] * items['unit_price'] orders['basket'] = orders['id'].map(items.groupby('order_id')['value'].sum()) cutoff = pd.Timestamp('2026-07-01') before = orders[orders['ordered_at'] < cutoff] after = orders[orders['ordered_at'] >= cutoff] people = before.groupby('customer_id').agg( orders=('id', 'size'), last=('ordered_at', 'max'), basket=('basket', 'mean')) people['days_since'] = (cutoff - people['last']).dt.days people['lost'] = (~people.index.isin(after['customer_id'])).astype(int) from sklearn.model_selection import train_test_split from sklearn.linear_model import LogisticRegression X = people[['orders', 'days_since', 'basket']] y = people['lost'] X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.25, random_state=42, stratify=y) model = LogisticRegression(max_iter=1000).fit(X_train, y_train) proba = model.predict_proba(X_test)[:, 1] top = ... hits = ... precision = ... recall = ... print(hits, precision, recall)

Check▶ Run

Marketing says: “We want to reach at least 80% of the lost customers.” Find in best the highest threshold from 0.05 to 0.5 where recall is at least 0.8, and in best_precision the precision at it.

Pythondata: the shop tables

import pandas as pd

orders = pd.read_csv('year_orders.csv', parse_dates=['ordered_at'])
items = pd.read_csv('year_items.csv')
items['value'] = items['quantity'] * items['unit_price']
orders['basket'] = orders['id'].map(items.groupby('order_id')['value'].sum())

cutoff = pd.Timestamp('2026-07-01')
before = orders[orders['ordered_at'] < cutoff]
after = orders[orders['ordered_at'] >= cutoff]

people = before.groupby('customer_id').agg(
    orders=('id', 'size'), last=('ordered_at', 'max'), basket=('basket', 'mean'))
people['days_since'] = (cutoff - people['last']).dt.days
people['lost'] = (~people.index.isin(after['customer_id'])).astype(int)

from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression

X = people[['orders', 'days_since', 'basket']]
y = people['lost']
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, random_state=42, stratify=y)
model = LogisticRegression(max_iter=1000).fit(X_train, y_train)
proba = model.predict_proba(X_test)[:, 1]
import numpy as np
from sklearn.metrics import precision_score, recall_score

best = None
for t in np.arange(0.05, 0.51, 0.05).round(2):
    pred = (proba >= t).astype(int)
    ...
best_precision = ...
print(best, best_precision)

import pandas as pd orders = pd.read_csv('year_orders.csv', parse_dates=['ordered_at']) items = pd.read_csv('year_items.csv') items['value'] = items['quantity'] * items['unit_price'] orders['basket'] = orders['id'].map(items.groupby('order_id')['value'].sum()) cutoff = pd.Timestamp('2026-07-01') before = orders[orders['ordered_at'] < cutoff] after = orders[orders['ordered_at'] >= cutoff] people = before.groupby('customer_id').agg( orders=('id', 'size'), last=('ordered_at', 'max'), basket=('basket', 'mean')) people['days_since'] = (cutoff - people['last']).dt.days people['lost'] = (~people.index.isin(after['customer_id'])).astype(int) from sklearn.model_selection import train_test_split from sklearn.linear_model import LogisticRegression X = people[['orders', 'days_since', 'basket']] y = people['lost'] X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.25, random_state=42, stratify=y) model = LogisticRegression(max_iter=1000).fit(X_train, y_train) proba = model.predict_proba(X_test)[:, 1] import numpy as np from sklearn.metrics import precision_score, recall_score best = None for t in np.arange(0.05, 0.51, 0.05).round(2): pred = (proba >= t).astype(int) ... best_precision = ... print(best, best_precision)

Check▶ Run

Test

Check yourself

1. The model is right about 72.2% of the test customers. The rule “everyone comes back” is right about 72.5%. What do you tell marketing?

The model is a bit weaker, but it is a model, so it is more reliableThe model is no better than the rule: of 88 lost customers it finds 1The model is good: 72% is a lot

2. What does precision measure when you give coupons to the customers the model thinks are lost?

Of the customers with a coupon, what share really would have leftWhat share of all customers the model gets rightOf everyone who would have left, what share gets a coupon

3. You lower the threshold from 0.4 to 0.25. What happens?

Nothing: the threshold does not change the modelThere are more coupons, recall goes up, precision goes down or staysThere are fewer and more precise coupons

4. Why are the features computed only from orders before 1 July?

So there is less data and training is fastBecause there are no orders after 1 JulyWhen you use the model, you only know the past: the label is from the future and the features must not see it

5. A coupon costs little, and a lost regular customer costs a lot. Which way do you lean?

Towards higher recall: a lower threshold and more couponsTowards higher accuracyTowards higher precision: a higher threshold and few coupons

Check

Start the lesson over

Related lessons

• Builds on: Dates and time series · The first chart · Models: learning from examples · Linear regression
• Continues in: Data leakage · Cross-validation
• Tasks on this topic: Models with scikit-learn