Skip to content
wannacode

Evaluation: accuracy lies

In short: A model is evaluated on data it has not seen: the confusion matrix counts the four outcomes, and precision and recall say more than accuracy.

Problem

99% accuracy, zero frauds caught

A colleague shows you their fraud model: “99% accuracy!” Sounds great. Here is a model that gets the same without doing anything:

const frauds = fraud.reduce((sum, y) => sum + y, 0);
// “always normal”: it never blocks, so it is right on every honest payment
const accuracy = 100 * (4000 - frauds) / 4000;
console.log(`payments: 4000, frauds: ${frauds}`);
console.log(`“always normal”: accuracy ${accuracy.toFixed(2)}%, frauds caught: 0 of ${frauds}`);
Output
payments: 4000, frauds: 40
“always normal”: accuracy 99.00%, frauds caught: 0 of 40

This is one month with 4000 payments, and 40 of them are fraud: one in a hundred. The “always normal” model blocks nothing. For 3960 honest payments it is right, for 40 frauds it is wrong. Accuracy 99.00%, frauds caught: zero.

Accuracy is the share of right answers. When one class is 99%, the model that always says that class is 99% accurate. The number is true, but it does not answer the shop’s question: “How many frauds do we catch, and how many honest customers do we annoy?”

See it

The month in code

Real payments come from the shop’s database. Here the month is made by a rule, so that it is small and the same in every language. Each payment is fraud with probability 1%. Frauds are usually 3× to 12× the usual amount, and sometimes a small test (0.3× to 1.3×). Honest payments are usually 0.2× to 2.2×, and sometimes a big purchase up to 6×.

The random numbers come from a small generator with a seed (an LCG): the same seed gives the same numbers in every language and on every run.

// a generator with a seed: the same numbers in every language
class Lcg {
  constructor(seed) {
    this.state = seed;
  }


  next() {
    this.state = (this.state * 1664525 + 1013904223) % 4294967296;
    return this.state / 4294967296;
  }
}
// one month of payments: about 1 in 100 is fraud
const rng = new Lcg(319);
const ratios = [];
const fraud = [];
for (let i = 0; i < 4000; i++) {
  const isFraud = rng.next() < 0.01;
  if (isFraud) // fraud: usually 3× to 12×, sometimes a small test
    ratios.push(rng.next() < 0.85 ? 3 + 9 * rng.next() : 0.3 + rng.next());
  else // honest: usually 0.2× to 2.2×, sometimes a big purchase
    ratios.push(rng.next() < 0.03 ? 2.5 + 3.5 * rng.next() : 0.2 + rng.next() + rng.next());
  fraud.push(isFraud ? 1 : 0);
}
The product state * 1664525 is below 2⁵³, so the number (a double) is exact, with no loss. The generator gives the same numbers as in the other languages.

The feature is the one from the classification lesson: the amount compared with the customer’s usual one.

The idea

Training and test

Before measuring anything, there is a more important question: on which data do we measure? Here is the simplest “model”: it remembers the amounts of the frauds it has seen and blocks only those. The month is shuffled and split: 70% for training, 30% for checking.

// shuffle (Fisher–Yates) with the same generator
const order = Array.from({ length: 4000 }, (_, i) => i);
for (let i = 3999; i > 0; i--) {
  const j = Math.floor(rng.next() * (i + 1));
  [order[i], order[j]] = [order[j], order[i]];
}
const train = order.slice(0, 2800);
const test = order.slice(2800);
console.log(`training: ${train.length} payments, ${countFrauds(fraud, train)} frauds`);
console.log(`test: ${test.length} payments, ${countFrauds(fraud, test)} frauds`);
// remembers the amounts of the frauds it has seen
const seen = new Set(train.filter((i) => fraud[i] === 1).map((i) => ratios[i]));
report("memorising on the training set", seen, ratios, fraud, train);
report("memorising on the test set", seen, ratios, fraud, test);
Output
training: 2800 payments, 24 frauds
test: 1200 payments, 16 frauds
memorising on the training set: accuracy 100.00%, caught 24 of 24
memorising on the test set: accuracy 98.67%, caught 0 of 16

On the payments it learned from, the model is perfect: 100%, 24 of 24. On new ones it catches no fraud at all, because a new fraud is never exactly the same amount as an old one. The model has learned nothing about frauds, it has only remembered them.

That is why the data is split in two before training:

  • training data (training set): the model learns from it;
  • test data (test set): it is used to measure at the end, and the model does not see it until then.

The test stands in for tomorrow’s payments. Shuffling before the split matters: if the month is sorted by date, the test will be only the last week.

See it

The confusion matrix

Now the real model: the logistic regression from the classification lesson. The function is the same, except that it also gets rows: which rows to use. It learns from the training data only:

// learns only from the rows in rows
function trainLogistic(xs, ys, rows, rate, epochs) {
  let w = 0, b = 0;
  const n = rows.length;
  for (let epoch = 0; epoch < epochs; epoch++) {
    let gw = 0, gb = 0;
    for (const i of rows) {
      const p = sigmoid(w * xs[i] + b);
      gw += (p - ys[i]) * xs[i];
      gb += p - ys[i];
    }
    w -= rate * gw / n;
    b -= rate * gb / n;
  }
  return { w, b };
}

A test payment can end in four ways. The model blocks or allows, and the payment is fraud or honest:

model blocks model allows
fraud blocked fraud (TP) missed fraud (FN)
honest blocked honest (FP) allowed honest (TN)

This is the confusion matrix. The letters stand for true/false positive/negative. “Positive” is the class we are looking for: fraud.

function confusion(ratios, fraud, rows, w, b,
    threshold) {
  let tp = 0, fp = 0, fn = 0, tn = 0;
  for (const i of rows) {
    const stop = block(ratios[i], w, b, threshold);
    if (stop && fraud[i] === 1) tp++; // blocked fraud
    else if (stop) fp++; // blocked honest customer
    else if (fraud[i] === 1) fn++; // missed fraud
    else tn++; // allowed honest customer
  }
  return { tp, fp, fn, tn };
}
const { w, b } = trainLogistic(ratios, fraud, train, 1, 1000);
const { tp, fp, fn, tn } =
  confusion(ratios, fraud, test, w, b, 0.5);
const accuracy = 100 * (tp + tn) / test.length;
const precision = 100 * tp / (tp + fp);
const recall = 100 * tp / (tp + fn);
console.log(`w = ${w.toFixed(4)}, b = ${b.toFixed(4)}`);
console.log(`frauds blocked (TP): ${tp}`);
console.log(`honest blocked (FP): ${fp}`);
console.log(`frauds missed (FN): ${fn}`);
console.log(`honest allowed (TN): ${tn}`);
console.log(`accuracy ${accuracy.toFixed(2)}%, precision ${precision.toFixed(2)}%, recall ${recall.toFixed(2)}%`);
Output
w = 1.0702, b = -7.1054
frauds blocked (TP): 7
honest blocked (FP): 0
frauds missed (FN): 9
honest allowed (TN): 1184
accuracy 99.25%, precision 100.00%, recall 43.75%

Accuracy is 99.25%, a little above the 98.67% of “always normal” on the same test. Yet the model catches 7 of 16 frauds and blocks no honest customer. The two accuracies are almost the same, and the models are completely different. We need numbers that look at the frauds.

See it

The matrix fills up

function confusion(ratios, fraud, rows, w, b,
    threshold) {
  let tp = 0, fp = 0, fn = 0, tn = 0;
  for (const i of rows) {
    const stop = block(ratios[i], w, b, threshold);
    if (stop && fraud[i] === 1) tp++; // blocked fraud
    else if (stop) fp++; // blocked honest customer
    else if (fraud[i] === 1) fn++; // missed fraud
    else tn++; // allowed honest customer
  }
  return { tp, fp, fn, tn };
}
Step by step, in words
  1. 1200 test payments the model has never seen, threshold 0.5. A row is the truth, a column is what the model said.
  2. First the 16 frauds, one by one: 7 blocked (TP), 9 missed (FN).
  3. Then the honest ones: 0 blocked (FP), 1184 allowed (TN). Almost everything ends up in one corner.
  4. Accuracy is the diagonal: (7 + 1184) / 1200 = 99.25%. “Always normal” gets 98.67% on the same test.
  5. Precision is the “block” column: of the blocked payments, how many were fraud? 7 / 7 = 100.00%.
  6. Recall is the “fraud” row: of the frauds, how many did we catch? 7 / 16 = 43.75%.
  7. Accuracy looks at everything and is high. Precision and recall look at the frauds: every block is a fraud, but we catch fewer than half.

1184 of the 1200 payments are in the bottom right corner: allowed honest. That is why accuracy is high for any model that lets honest payments through. The interesting part is in the top row.

The idea

Precision and recall in the words of the shop

Two questions, two numbers:

  • Precision: of the blocked payments, how many really are fraud? TP / (TP + FP). Low precision means many annoyed honest customers.
  • Recall: of all frauds, how many did we catch? TP / (TP + FN). Low recall means missed frauds and lost money.

At threshold 0.5 the model has precision 100% and recall 43.75%. Every block is a fraud, but it catches fewer than half. For “always normal” recall is 0%, and you see that at once, without fooling yourself with 99%.

The two pull against each other. To catch more frauds (recall), you block more suspicious payments, and some of them are honest (precision falls).

Try it

Fill in the blank

Precision divides by all blocked payments. What does recall divide by?

const precision = 100 * tp / (tp + fp);
const recall = 100 * tp / (tp + );

Try it

Predict the output

Ten payments: the truth (1 = fraud) and what the model blocked (1 = blocked). What will it print?

const actual = [1, 0, 0, 1, 0, 0, 0, 1, 0, 0];
const blocked = [1, 1, 0, 0, 0, 1, 0, 1, 0, 0];
let tp = 0, fp = 0, fn = 0;
for (let i = 0; i < actual.length; i++) {
  if (blocked[i] === 1 && actual[i] === 1) tp++;
  else if (blocked[i] === 1) fp++;
  else if (actual[i] === 1) fn++;
}
console.log(`precision ${(100 * tp / (tp + fp)).toFixed(2)}%, recall ${(100 * tp / (tp + fn)).toFixed(2)}%`);
Answer

The idea

The threshold and the cost in euro

The threshold from the classification lesson now has numbers. The same model on the same test, at four thresholds:

for (const threshold of [0.5, 0.3, 0.1, 0.02]) {
  const { tp, fp, fn } = confusion(ratios, fraud, test, w, b, threshold);
  const precision = 100 * tp / (tp + fp);
  const recall = 100 * tp / (tp + fn);
  console.log(`threshold ${threshold.toFixed(2)}: TP ${tp}, FP ${fp}, FN ${fn} → precision ${precision.toFixed(2)}%, recall ${recall.toFixed(2)}%`);
}
Output
threshold 0.50: TP 7, FP 0, FN 9 → precision 100.00%, recall 43.75%
threshold 0.30: TP 10, FP 1, FN 6 → precision 90.91%, recall 62.50%
threshold 0.10: TP 11, FP 13, FN 5 → precision 45.83%, recall 68.75%
threshold 0.02: TP 14, FP 28, FN 2 → precision 33.33%, recall 87.50%

The lower the threshold, the more frauds we catch (recall grows from 43.75% to 87.5%) and the more honest customers we block (precision falls from 100% to 33.33%). Which is better? The answer is in euro. Say a missed fraud costs €250 (the goods and the chargeback fee), and a blocked honest customer costs €10 (a call to support and lost time). The prices are an example:

// example prices: a missed fraud and a blocked honest customer
const missedCost = 250, blockedCost = 10;
let best = -1;
let bestCost = Infinity;
for (const threshold of [0.5, 0.3, 0.1, 0.02]) {
  const { fp, fn } = confusion(ratios, fraud, test, w, b, threshold);
  const cost = fn * missedCost + fp * blockedCost;
  console.log(`threshold ${threshold.toFixed(2)}: ${fn} × €${missedCost} + ${fp} × €${blockedCost} = €${cost}`);
  if (cost < bestCost) {
    bestCost = cost;
    best = threshold;
  }
}
console.log(`cheapest threshold: ${best.toFixed(2)}`);
Output
threshold 0.50: 9 × €250 + 0 × €10 = €2250
threshold 0.30: 6 × €250 + 1 × €10 = €1510
threshold 0.10: 5 × €250 + 13 × €10 = €1380
threshold 0.02: 2 × €250 + 28 × €10 = €780
cheapest threshold: 0.02

With these prices the strictest threshold is the cheapest: 0.02, with €780 instead of €2250 at 0.5. One missed fraud costs as much as 25 blocked customers, so it pays to block more often. If a blocked customer leaves for good and costs much more, the answer is different. That is the exercise at the end.

One warning. Here the threshold is picked with the test. So the test has already taken part in a decision, and the €780 is a little more optimistic than the truth. In a real project the threshold is picked with a third share of the data, the validation set, and the test is run once, at the end. How, is in the next lesson.

Try it

Find the bug

A colleague splits the data like this. Their test numbers are excellent, and in the shop the model catches far fewer. On which line is the bug?

Pick the line with the bug.

The idea

When it hurts

  • Accuracy with uneven classes. Fraud, illness, breakdowns, spam: when the thing you look for is rare, accuracy is almost useless. Look at the matrix, precision and recall.
  • A test that looks more like the training data than like the future. Fraudsters change their tricks. If the test is shuffled with past months, it is easier than next month. For such data the test is the newer period: you learn up to 30 September and measure October.
  • Few examples of the rare class. The test has 16 frauds. One fraud more or less moves recall by 6.25%. The numbers here are for the lesson, and real teams collect thousands of frauds before they trust the second digit.
  • One number for everything. Precision and recall are two because there are two mistakes and they cost different amounts. If you squash them into one, you lose the decision about the cost.
  • A test touched many times. Every time you change the model because the test said “bad”, the test becomes a little bit of training.
More precisely, if you are curious

F1. The harmonic mean of precision and recall: 2 · P · R / (P + R). One number that falls if either of the two is low. It is useful for comparing, but it does not know the cost of the mistakes.

The curve. If you move the threshold from 1 to 0 and plot precision against recall, you get the PR curve. The area under it (PR AUC) measures the model without a specific threshold. The related ROC curve (share of frauds caught against share of honest blocked) is better known, but at 1 in 100 it looks better than it is, because there are so many honest payments.

The threshold from the cost. If the model’s probabilities are right, the cheapest threshold is the cost of a blocked customer divided by the sum of the two costs: 10 / (10 + 250) ≈ 0.04. The program found 0.02 among the four thresholds it tried. The model here is learned on 1% fraud, so its probabilities are close to the real ones. A model learned on 40% fraud (as in the classification lesson) sees fraud too often, and its threshold has to be higher.

Splitting by class. With 40 frauds a random split can send 30 to training and 10 to the test, or the other way round. A stratified split keeps the same share of frauds in both parts.

In real code. In scikit-learn these are train_test_split (with stratify= for the classes), confusion_matrix, precision_score, recall_score and precision_recall_curve. In ML.NET TrainTestSplit and BinaryClassificationMetrics give the same. Watching the model in production (whether recall drops month after month) is part of MLOps and is taught in ZeroToOps.

Set keeps numbers as keys: seen.has(3.7) is true only for exactly the same number.

Exercise

The expensive honest customer

Solve this in your editor.

The shop sells subscriptions. A customer blocked by mistake often goes to a competitor, so a blocked honest customer costs €60, and a missed fraud still costs €250. Run the cost program with €60 and find the cheapest threshold.

Output
threshold 0.50: 9 × €250 + 0 × €60 = €2250
threshold 0.30: 6 × €250 + 1 × €60 = €1560
threshold 0.10: 5 × €250 + 13 × €60 = €2030
threshold 0.02: 2 × €250 + 28 × €60 = €2180
cheapest threshold: 0.30

Now the cheapest is 0.3: it blocks 10 of 16 frauds and only 1 honest customer. The model is the same, the data is the same, and the threshold is different, because the mistakes cost different amounts.

More tasks

  • A finer grid of thresholds. Try the thresholds from 0.02 to 0.5 in steps of 0.02. Is there a cheaper threshold than 0.3?
  • Test by time. Instead of shuffling, take the first 2800 payments for training and the last 1200 for the test. How many frauds are in the test now, and how did recall change at threshold 0.3?

Test

Check yourself

1. Frauds are 1 in 100. A model says “normal” for every payment. What is its accuracy?
2. Why is a model checked on data it has not seen during training?
3. Precision is 90%. What does that mean for the shop?
4. A missed fraud costs €250, and a blocked honest customer costs €10. Which way should the threshold move?
5. A colleague picks the threshold with the lowest cost on the test and reports that cost. What is wrong?

Made by Ivo Gergov as part of Evo Tech's campaign for accessible programming education ·