Skip to content
wannacode

Tests with unittest

Problem

The discount that broke quietly

The shop's rule is: the discount is in percent, and on a purchase of €100 or more you add another 5%. You write the function and try it by hand:

main.py
def apply_discount(price, percent):
    # Discount in percent. From €100 up, add another 5%.
    if price > 100:
        percent = percent + 5
    return price * (100 - percent) / 100




print(apply_discount(50, 10))
print(apply_discount(200, 10))
Output
45.0
170.0

€50 with 10% off is €45, and €200 with 15% off is €170. It looks right, so you ship it. A week later a customer with a purchase of exactly €100 pays €90 instead of €85. The condition is price > 100, and it should be price >= 100.

Nobody tried exactly €100. And on every later change to the function, you have to try all the cases by hand again.

The idea

Code that checks code

Instead of reading the output with your own eyes, you write small methods that call your code and compare the result with the expected one. That is a test. You run all the tests with one click, after every change, and they tell you what broke.

Kind What it checks Example
Unit test one small piece, usually one function, apart from the rest apply_discount(100, 10) returns 85
Integration how the pieces work together: with a database, a file, another service the order is saved to the database and read back
End-to-end the whole application, the way a person uses it the customer adds to the cart, pays and gets an email
In this lesson we write unit tests. Python has the unittest module for them in the standard library: there is nothing to install.

Why it is worth it:

  • you see the bug right away, while you still remember what you changed, not when the customer complains;
  • the test shows how the code is used: which case returns what;
  • you can clean up the code boldly. If the tests are still green after the change, the behavior has not changed.
More precisely, if you are curious

There are other kinds of tests too: functional (does the program meet the requirements), performance (how many requests it can handle), security (are there any holes) and user interface (the buttons, the forms). All of them build on unit tests: there are the most of them, and they are the fastest and the cheapest to maintain.

Besides unittest, you will often see pytest in Python: a separate library (pip install pytest) where tests are plain functions with assert. Here we use unittest, because it comes with Python itself.

See it

The first test

apply_discount is in the file discount.py. Its tests are in a separate file, test_discount.py, in the class DiscountTest:
test_discount.py
import unittest


from discount import apply_discount




class DiscountTest(unittest.TestCase):
    def test_ten_percent_off(self):
        self.assertEqual(apply_discount(50, 10), 45.0)


    def test_extra_five_percent_from_100(self):
        self.assertEqual(apply_discount(100, 10), 85.0)
  • The test class inherits from unittest.TestCase. Every method whose name starts with test is a test: unittest finds it on its own by the name. How exactly, you will see in the last lesson of the module.
  • self.assertEqual(a, b) compares the two values. If they are different, the test fails and says a != b. Here we write what the code returned first, and then the expected value.

We run the two tests against the code from the first card:

Result
FAIL  test_extra_five_percent_from_100: 90.0 != 85.0
PASS  test_ten_percent_off
Tests: 2, passed: 1, failed: 1

The first test passes. The second one fails and says exactly what happened: it expected 85.0, and the function returned 90.0. You fix the condition to price >= 100, run them again, and both are green:

Result
PASS  test_extra_five_percent_from_100
PASS  test_ten_percent_off
Tests: 2, passed: 2, failed: 0

Every test does three things: it prepares the values, calls the code and checks the result.

class DiscountTest(unittest.TestCase):
    def test_ten_percent_off(self):
        self.assertEqual(apply_discount(50, 10), 45.0)


    def test_extra_five_percent_from_100(self):
        self.assertEqual(apply_discount(100, 10), 85.0)
Step by step, in words
  1. DiscountTest has two tests. unittest runs each one on its own and lights a lamp next to it.
  2. Test 1, prepare: a price of €50 and 10% off.
  3. Act: apply_discount(50, 10) returns 45.0.
  4. Check: assertEqual compares the expected 45.0 with the returned 45.0. They are the same, so the test passes.
  5. Test 2: exactly €100 and 10%. We expect 85.0, because from €100 up there is another 5%.
  6. The condition in the function is price > 100, and 100 is not greater than 100. The function returns 90.0.
  7. Expected 85.0, returned 90.0: the test fails and says exactly that.
  8. You fix the condition to price >= 100 and run the tests again.
  9. Now 100 passes the condition and the function returns 85.0.
  10. Expected and returned match. Both tests are green.
More precisely, if you are curious
  • In the project the code and the tests are files side by side. Test files start with test, so unittest can find them:
Layout
discount.py          the code
test_discount.py     the tests for it
  • You run them from the terminal. Without -v, unittest prints a dot for each test that passed and F for each one that failed, and then the details of the failed ones:
Commands
python -m unittest            finds test*.py and runs them
python -m unittest -v         with a line for each test
python -m unittest test_discount.DiscountTest.test_ten_percent_off
  • Here, on the site, the code and the tests are in one file, and the tests are run through unittest's programming interface: a custom TestResult collects the result of each test, and run_tests prints them sorted by name. Sorting keeps the output the same on every run:
The runner
def name_of(test):
    # The docstring of the test or the name of the method (with the subTest case).
    return test.shortDescription() or test.id().split(".", 2)[2]




class Report(unittest.TestResult):
    def __init__(self):
        super().__init__()
        self.lines = []
        self.failed = 0
        self.with_cases = set()


    def passed(self, name):
        self.lines.append(f"PASS  {name}")


    def fell(self, name, err):
        self.failed += 1
        self.lines.append(f"FAIL  {name}: {err[1]}")


    def addSuccess(self, test):
        if test.id() not in self.with_cases:
            self.passed(name_of(test))


    def addFailure(self, test, err):
        self.fell(name_of(test), err)


    addError = addFailure


    def addSubTest(self, test, subtest, err):
        self.with_cases.add(test.id())
        name = subtest.id().split(".", 2)[2]
        if err is None:
            self.passed(name)
        else:
            self.fell(name, err)




def run_tests(*tests):
    loader = unittest.TestLoader()
    suite = unittest.TestSuite()
    for t in tests:
        suite.addTest(t if isinstance(t, unittest.TestSuite) else loader.loadTestsFromTestCase(t))
    report = Report()
    suite.run(report)
    for line in sorted(report.lines):
        print(line)
    total = len(report.lines)
    print(f"Tests: {total}, passed: {total - report.failed}, failed: {report.failed}")

Try it

VAT with float

The price with VAT is the price times 1.2. Two tests:

What will the tests show?

def with_vat(price):
    return price * 1.2




class VatTest(unittest.TestCase):
    def test_coffee(self):
        self.assertEqual(with_vat(10), 12.0)


    def test_tea(self):
        self.assertEqual(with_vat(3), 3.6)
Answer

The idea

Yes or no, and an expected exception

Besides equality, you often check a condition, or that the code refuses. Here is Account from the module on exceptions, with a new method, can_pay:

account.py
class InsufficientFundsError(Exception):
    def __init__(self, shortfall):
        super().__init__(f"Short by €{shortfall}")




class Account:
    def __init__(self, balance):
        self._balance = balance


    @property
    def balance(self):
        return self._balance


    def can_pay(self, amount):
        return amount <= self._balance


    def withdraw(self, amount):
        if amount <= 0:
            raise ValueError("The amount must be positive")
        if amount > self._balance:
            raise InsufficientFundsError(amount - self._balance)
        self._balance -= amount
test_account.py
import unittest


from account import Account, InsufficientFundsError




class AccountTest(unittest.TestCase):
    def test_can_pay_what_is_there(self):
        account = Account(100)
        self.assertTrue(account.can_pay(100))
        self.assertFalse(account.can_pay(101))


    def test_withdraw_too_much(self):
        account = Account(100)
        with self.assertRaises(InsufficientFundsError) as caught:
            account.withdraw(130)
        self.assertEqual(str(caught.exception), "Short by €30")
        self.assertEqual(account.balance, 100)
Output
PASS  test_can_pay_what_is_there
PASS  test_withdraw_too_much
Tests: 2, passed: 2, failed: 0
  • assertTrue and assertFalse check a condition.
  • with self.assertRaises(Type): fails if the code in the block does not raise an exception of that type. With as caught you get the exception itself: caught.exception, and str() of it is the message.

The second test also checks something that is easy to forget: after the refusal the balance is the same.

The idea

Every test starts clean

Here the two tests share one account with €100:

test_account.py
class AccountTest(unittest.TestCase):
    account = Account(100)


    def test_withdraw_40(self):
        """withdrawing €40 leaves €60"""
        self.account.withdraw(40)
        self.assertEqual(self.account.balance, 60)


    def test_withdraw_70(self):
        """withdrawing €70 leaves €30"""
        self.account.withdraw(70)
        self.assertEqual(self.account.balance, 30)
Output
FAIL  withdrawing €70 leaves €30: Short by €10
PASS  withdrawing €40 leaves €60
Tests: 2, passed: 1, failed: 1

account is a class attribute: one object for all the tests. unittest runs the tests in alphabetical order of their names, so the test for €40 is first. The second one has €60 left, and it withdraws €70. All it takes is for someone to rename a test or add a new one, and a different test fails. A test that depends on another test is a trap. So the account is created again before each test, in the setUp method:

test_account.py
import unittest


from account import Account




class AccountTest(unittest.TestCase):
    def setUp(self):
        self.account = Account(100)


    def test_withdraw_40(self):
        """withdrawing €40 leaves €60"""
        self.account.withdraw(40)
        self.assertEqual(self.account.balance, 60)


    def test_withdraw_70(self):
        """withdrawing €70 leaves €30"""
        self.account.withdraw(70)
        self.assertEqual(self.account.balance, 30)
Output
PASS  withdrawing €40 leaves €60
PASS  withdrawing €70 leaves €30
Tests: 2, passed: 2, failed: 0
unittest makes a new object of the class for each test and calls setUp before it, so self.account is a new account every time. The string under def (the docstring) describes the test in words: its first line is shown in the result instead of the method name.
More precisely, if you are curious
More things from unittest you will come across:
unittest
setUp / tearDown                before / after each test (cleanup)
setUpClass / tearDownClass      once for the whole class (@classmethod)
@unittest.skip("...")           the test is skipped for now
@unittest.skipIf(condition, "...") skipped if the condition is true
@unittest.expectedFailure       it is expected to fail
with self.subTest(...):         a separate case inside one test
self.addCleanup(function)       cleanup after the test, even if it failed
python -m unittest -k withdraw  only the tests with "withdraw" in the name
python -m unittest -f           stops at the first failed test

The idea

One check, many cases

Four cases for the discount mean four almost identical methods. Instead, you write one method with a loop over the cases "price, percent, expected". with self.subTest(...) makes each case a separate check:

test_discount.py
import unittest


from discount import apply_discount




class DiscountTest(unittest.TestCase):
    def test_apply(self):
        cases = [
            (50, 10, 45.0),
            (99, 0, 99.0),
            (100, 10, 85.0),
            (200, 20, 150.0),
        ]
        for price, percent, expected in cases:
            with self.subTest(f"€{price} with {percent}% off = €{expected}"):
                self.assertEqual(apply_discount(price, percent), expected)
Output
PASS  test_apply [€100 with 10% off = €85.0]
PASS  test_apply [€200 with 20% off = €150.0]
PASS  test_apply [€50 with 10% off = €45.0]
PASS  test_apply [€99 with 0% off = €99.0]
Tests: 4, passed: 4, failed: 0

Each case has a separate result, and the text in subTest is its name. Without subTest, the first failed case stops the whole method, and you cannot tell whether the others pass. A new case is a new row in the list, not a new method. The important cases are at the edges: exactly €100, €99, 0%.

Try it

Find the bug from the test

Delivery is €4, and free from €50 up. The test:

test_delivery.py
class DeliveryTest(unittest.TestCase):
    def test_price(self):
        for total, expected in [(20, 4), (50, 0), (80, 0)]:
            with self.subTest(f"order of €{total}: delivery €{expected}"):
                self.assertEqual(delivery_price(total), expected)
Output
FAIL  test_price [order of €50: delivery €0]: 4 != 0
PASS  test_price [order of €20: delivery €4]
PASS  test_price [order of €80: delivery €0]
Tests: 3, passed: 2, failed: 1

Which line of the code is the bug on?

Pick the line with the bug.

The idea

The test first

The checkout has to accept the promo code FALL10 for 10% off, whether it is in lowercase or uppercase. This time you write the tests first:

test_promo.py
class PromoTest(unittest.TestCase):
    def test_known_code(self):
        self.assertEqual(promo_percent("FALL10"), 10)


    def test_code_is_case_insensitive(self):
        self.assertEqual(promo_percent("fall10"), 10)


    def test_unknown_code(self):
        self.assertEqual(promo_percent("NOPE"), 0)
  1. Red. For now the function returns 0. The tests fail, and that is good: it means they really check something.
def promo_percent(code):
    return 0
Output
FAIL  test_code_is_case_insensitive: 0 != 10
FAIL  test_known_code: 0 != 10
PASS  test_unknown_code
Tests: 3, passed: 1, failed: 2
  1. Green. You write the simplest code that makes them pass:
def promo_percent(code):
    if code.upper() == "FALL10":
        return 10
    return 0
Output
PASS  test_code_is_case_insensitive
PASS  test_known_code
PASS  test_unknown_code
Tests: 3, passed: 3, failed: 0
  1. Clean up. Now you can move the codes into a dictionary (dict), add new ones and clean up the function. The tests will tell you if you break something.

This is TDD (test-driven development). Writing the test first makes you decide what exactly the function has to do, before you think about how.

More precisely

Suites and fake objects

A suite (unittest.TestSuite) is a group of tests that run together, for example all the tests of the checkout. The function builds it from the three test classes, and TextTestRunner runs it:

test_checkout.py
def checkout_suite():
    loader = unittest.TestLoader()
    suite = unittest.TestSuite()
    suite.addTests(loader.loadTestsFromTestCase(DiscountTest))
    suite.addTests(loader.loadTestsFromTestCase(AccountTest))
    suite.addTests(loader.loadTestsFromTestCase(DeliveryTest))
    return suite




unittest.TextTestRunner().run(checkout_suite())
Output
PASS  test_extra_five_percent_from_100
PASS  test_price [order of €20: delivery €4]
PASS  test_price [order of €50: delivery €0]
PASS  test_price [order of €80: delivery €0]
PASS  test_ten_percent_off
PASS  withdrawing €40 leaves €60
PASS  withdrawing €70 leaves €30
Tests: 7, passed: 7, failed: 0

The output is from the site's runner. python -m unittest does the same without a suite: it collects the tests from all the test*.py files on its own.

More precisely, if you are curious
  • Sometimes the code depends on something you do not want to call in a test: an exchange rate from a bank, a payment, an email. Then you pass in a fake object (a mock) that returns whatever you tell it to. In Python it is in the standard library: unittest.mock.Mock:
test_converter.py
import unittest
from unittest.mock import Mock




class Converter:
    def __init__(self, rates):
        self.rates = rates


    def to_usd(self, eur):
        return eur * self.rates.eur_to("USD")




class PriceInUsdTest(unittest.TestCase):
    def test_converts_with_todays_rate(self):
        rates = Mock()
        rates.eur_to.return_value = 1.10


        converter = Converter(rates)
        self.assertAlmostEqual(converter.to_usd(10), 11.0, delta=0.001)
        rates.eur_to.assert_called_once_with("USD")
Output
PASS  test_converts_with_todays_rate
Tests: 1, passed: 1, failed: 0

Mock() is an object that accepts any method. return_value says what eur_to returns, and assert_called_once_with checks that it was called exactly once, with "USD". You do not need an interface: Converter calls eur_to on whatever you pass it (duck typing).

  • More checks besides assertEqual, assertTrue and assertRaises:
Checks
assertNotEqual(a, b)             are they different
assertIs(a, b), assertIsNot(a, b)  is it the same object (is), not just equal
assertIsNone(x), assertIsNotNone(x)
assertIn(x, items), assertNotIn(x, items)
assertCountEqual(a, b)           the same elements, in any order
assertListEqual(a, b)            two lists with the same elements in the same order
assertRegex(text, pattern)       the text matches a regular expression
assertGreater(a, b), assertLess(a, b)
assertAlmostEqual(a, b, delta=0.01)
self.fail("message")             the test fails right away

Exercise

Write the tests

Solve these in your editor, with unittest (it comes with Python itself).

1. The discount calculator. Write test_discount.py for apply_discount from this lesson. Cover these cases: €80 with 10% off is €72, €100 with 0% off is €95, €150 with 20% off is €112.50, €0 with 30% off is €0, and exactly €100 with 10% off is €85. Run them against the broken version with price > 100 too. Exactly the two cases with €100 should fail:

Result against the broken version
FAIL  test_apply [€100 with 0% off = €95.0]: 100.0 != 95.0 within 0.001 delta (5.0 difference)
FAIL  test_exactly_100_gets_the_extra: 90.0 != 85.0 within 0.001 delta (5.0 difference)
PASS  test_apply [€0 with 30% off = €0.0]
PASS  test_apply [€150 with 20% off = €112.5]
PASS  test_apply [€80 with 10% off = €72.0]
Tests: 5, passed: 3, failed: 2

2. The account. Write AccountTest for Account from the card "Yes or no, and an expected exception". Create the account with €100 in setUp. Check that:

  • withdrawing €30 leaves €70;
  • you can withdraw everything, and 0 is left;
  • withdrawing €150 throws InsufficientFundsError with the message "Short by €50", and the balance stays €100;
  • 0 and -20 throw ValueError (one test with subTest for the two amounts).

3. The bug in the average grade. The function looks fine:

grades.py
def average(grades):
    total = 0
    for grade in grades:
        total += grade
    return total // len(grades)
Write two tests: the average of 4 and 6 is 5.0, and the average of 5 and 6 is 5.5 (with an allowed difference of 0.001). One of them fails:
Result
FAIL  test_average_with_a_fraction: 5 != 5.5 within 0.001 delta (0.5 difference)
PASS  test_whole_average
Tests: 2, passed: 1, failed: 1

Find the cause and fix the function, so that both pass.

Test

Check yourself

1. What does a unit test check?
2. self.assertEqual(apply_discount(100, 10), 85.0) fails with "90.0 != 85.0". What did the function return?
3. Why is the account created in setUp and not once as a class attribute?
4. How do you check that withdraw(130) raises InsufficientFundsError?
5. What is the order in TDD?

Made by Ivo Gergov as part of Evo Tech's campaign for accessible programming education ·