Component tests

In short: A scenario is for the examples the business agreed on and wants to read. A component test calls the code directly: ten amounts in milliseconds, with no steps and no Gherkin. Both check one rule for different readers. The scenarios stay few, and the component tests take on the technical cases.

Problem

Fifteen scenarios for one boundary

The programmer decided to check delivery properly: 1 cent, 10 €, 49.49 €, 49.50 €, 49.99 €, 50 €, 50.01 €, 100 €, 250 €, a negative amount. He wrote everything down as scenarios.

The shop owner opened the scenarios file to see the delivery rule. She saw fifteen nearly identical scenarios and closed the file. The living documentation was no longer being read.

The idea

The test calls the code directly

From the programming course you know component tests: a method that calls the code and compares the result. The delivery rule needs neither Gherkin nor steps. The test calls the method directly:

Java

src/test/java/shop/DeliveryTest.java

@ParameterizedTest(name = "feeFor({0}) = {1}")
@CsvSource({ "4999, 490", "5000, 0", "1, 490", "25000, 0" })
void feeForTotal(int total, int fee) {
    assertEquals(fee, Delivery.feeFor(total));
}

C#

Shop.Tests/DeliveryTest.cs

[Theory]
[InlineData(4999, 490)]
[InlineData(5000, 0)]
[InlineData(1, 490)]
[InlineData(25000, 0)]
public void FeeFor(int total, int fee)
{
    Assert.Equal(fee, Delivery.FeeFor(total));
}

Python

test_delivery.py

@pytest.mark.parametrize("total, fee", [(4999, 490), (5000, 0), (1, 490), (25000, 0)])
def test_fee_for_total(total, fee):
    assert delivery_fee(total) == fee

JavaScript

delivery.test.js

const cases = [[4999, 490], [5000, 0], [1, 490], [25000, 0]];
for (const [total, fee] of cases) {
  test(`deliveryFee(${total}) = ${fee}`, () => {
    assert.equal(deliveryFee(total), fee);
  });
}

This is a parameterised test: one check and a table of cases. It looks like a scenario outline with "Examples", but it is for the programmer. A new case is a new row in the table.

The refusal is checked with a separate test. The rule throws an exception, and the test checks that it does:

Java

src/test/java/shop/DeliveryTest.java

@Test
@DisplayName("an empty cart is refused")
void emptyCartIsRefused() {
    assertThrows(IllegalArgumentException.class, () -> Delivery.feeFor(0));
}

C#

Shop.Tests/DeliveryTest.cs

[Fact(DisplayName = "an empty cart is refused")]
public void EmptyCartIsRefused()
{
    Assert.Throws<ArgumentException>(() => Delivery.FeeFor(0));
}

Python

test_delivery.py

def test_empty_cart_is_refused():
    with pytest.raises(ValueError):
        delivery_fee(0)

JavaScript

delivery.test.js

test("an empty cart is refused", () => {
  assert.throws(() => deliveryFee(0));
});

More precisely, if you are curious

The tools are the standard ones for each language: JUnit 5 (parameterised tests live in the junit-jupiter-params module), xUnit with [Theory] and [InlineData], pytest, and node:test, built into Node. In the programming course the Python tests used unittest. pytest is the more widespread one: a test is an ordinary function with assert, and parametrize builds the table.

The idea

Which rule on which level

The rule is one. The checks are on two levels, and each has its own reader:

• A scenario is for the examples the business agreed on: exactly 50.00 €, 49.99 €, an empty cart. One or two examples per rule. It is the living documentation, and the business reads it.
• A component test is for the technical cases: 1 cent, a large amount, a negative amount, rounding. They protect the code, but they say nothing new about the rule.

Checking the boundary on both levels is not a waste. The scenario says what was agreed. The component test fails first, and exactly where the bug is.

The rule for choosing is short: would the business want to read this example? If yes, a scenario. If not, a component test.

Try it

Scenario or component test?

1. "A cart of exactly 50.00 €: delivery is free." The business agreed on it at the story meeting.

Only a component testA scenario: it is the agreed example for the boundary, and the business has to be able to read itA UI test

2. An amount of 1 cent, an amount of 250 € and an amount of 99,999.99 €.

A manual checkOne scenario for eachComponent tests: technical cases the business does not want to read

3. A negative amount, say -5.00 €, which no customer can enter.

It is not testedA component test: it checks that the code protects itself from impossible input tooA scenario for the business

Check

The idea

The run

The whole test file, run against the correct code. Everything passes:

Java

src/test/java/shop/DeliveryTest.java

package shop;

import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.CsvSource;

import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertThrows;

class DeliveryTest {
    @ParameterizedTest(name = "feeFor({0}) = {1}")
    @CsvSource({ "4999, 490", "5000, 0", "1, 490", "25000, 0" })
    void feeForTotal(int total, int fee) {
        assertEquals(fee, Delivery.feeFor(total));
    }

    @Test
    @DisplayName("an empty cart is refused")
    void emptyCartIsRefused() {
        assertThrows(IllegalArgumentException.class, () -> Delivery.feeFor(0));
    }
}

Output

PASS  an empty cart is refused
PASS  feeFor(1) = 490
PASS  feeFor(25000) = 0
PASS  feeFor(4999) = 490
PASS  feeFor(5000) = 0
Tests: 5, passed: 5, failed: 0

C#

Shop.Tests/DeliveryTest.cs

using Xunit;

namespace Shop;

public class DeliveryTest
{
    [Theory]
    [InlineData(4999, 490)]
    [InlineData(5000, 0)]
    [InlineData(1, 490)]
    [InlineData(25000, 0)]
    public void FeeFor(int total, int fee)
    {
        Assert.Equal(fee, Delivery.FeeFor(total));
    }

    [Fact(DisplayName = "an empty cart is refused")]
    public void EmptyCartIsRefused()
    {
        Assert.Throws<ArgumentException>(() => Delivery.FeeFor(0));
    }
}

Output

PASS  FeeFor(total: 1, fee: 490)
PASS  FeeFor(total: 25000, fee: 0)
PASS  FeeFor(total: 4999, fee: 490)
PASS  FeeFor(total: 5000, fee: 0)
PASS  an empty cart is refused
Tests: 5, passed: 5, failed: 0

Python

test_delivery.py

import pytest

from delivery import delivery_fee


@pytest.mark.parametrize("total, fee", [(4999, 490), (5000, 0), (1, 490), (25000, 0)])
def test_fee_for_total(total, fee):
    assert delivery_fee(total) == fee


def test_empty_cart_is_refused():
    with pytest.raises(ValueError):
        delivery_fee(0)

Output

PASS  test_empty_cart_is_refused
PASS  test_fee_for_total[1-490]
PASS  test_fee_for_total[25000-0]
PASS  test_fee_for_total[4999-490]
PASS  test_fee_for_total[5000-0]
Tests: 5, passed: 5, failed: 0

JavaScript

delivery.test.js

import { test } from "node:test";
import assert from "node:assert/strict";
import { deliveryFee } from "./delivery.js";

const cases = [[4999, 490], [5000, 0], [1, 490], [25000, 0]];
for (const [total, fee] of cases) {
  test(`deliveryFee(${total}) = ${fee}`, () => {
    assert.equal(deliveryFee(total), fee);
  });
}

test("an empty cart is refused", () => {
  assert.throws(() => deliveryFee(0));
});

Output

PASS  an empty cart is refused
PASS  deliveryFee(1) = 490
PASS  deliveryFee(25000) = 0
PASS  deliveryFee(4999) = 490
PASS  deliveryFee(5000) = 0
Tests: 5, passed: 5, failed: 0

Now the programmer "simplifies" the comparison and rounds the amount to whole euros before it. 49.99 € becomes 50 €, and delivery becomes free:

Java

src/main/java/shop/Delivery.java

package shop;

public class Delivery {
    public static final int FREE_FROM = 5000;
    public static final int FEE = 490;

    // The delivery fee in cents. total is the cart total, also in cents.
    public static int feeFor(int total) {
        if (total <= 0) {
            throw new IllegalArgumentException("An empty cart cannot be ordered");
        }
        return Math.round(total / 100.0) * 100 >= FREE_FROM ? 0 : FEE;
    }
}

C#

Shop/Delivery.cs

namespace Shop;

public static class Delivery
{
    public const int FreeFrom = 5000;
    public const int Fee = 490;

    // The delivery fee in cents. total is the cart total, also in cents.
    public static int FeeFor(int total)
    {
        if (total <= 0)
        {
            throw new ArgumentException("An empty cart cannot be ordered");
        }
        return Math.Round(total / 100.0) * 100 >= FreeFrom ? 0 : Fee;
    }
}

Python

delivery.py

FREE_FROM = 5000
FEE = 490


def delivery_fee(total):
    """The delivery fee in cents. total is the cart total, also in cents."""
    if total <= 0:
        raise ValueError("An empty cart cannot be ordered")
    return 0 if round(total / 100) * 100 >= FREE_FROM else FEE

JavaScript

delivery.js

export const FREE_FROM = 5000;
export const FEE = 490;

// The delivery fee in cents. total is the cart total, also in cents.
export function deliveryFee(total) {
  if (total <= 0) {
    throw new Error("An empty cart cannot be ordered");
  }
  return Math.round(total / 100) * 100 >= FREE_FROM ? 0 : FEE;
}

The same tests, against the new code:

Java

src/test/java/shop/DeliveryTest.java

package shop;

import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.CsvSource;

import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertThrows;

class DeliveryTest {
    @ParameterizedTest(name = "feeFor({0}) = {1}")
    @CsvSource({ "4999, 490", "5000, 0", "1, 490", "25000, 0" })
    void feeForTotal(int total, int fee) {
        assertEquals(fee, Delivery.feeFor(total));
    }

    @Test
    @DisplayName("an empty cart is refused")
    void emptyCartIsRefused() {
        assertThrows(IllegalArgumentException.class, () -> Delivery.feeFor(0));
    }
}

Output

FAIL  feeFor(4999) = 490: expected: <490> but was: <0>
PASS  an empty cart is refused
PASS  feeFor(1) = 490
PASS  feeFor(25000) = 0
PASS  feeFor(5000) = 0
Tests: 5, passed: 4, failed: 1

C#

Shop.Tests/DeliveryTest.cs

using Xunit;

namespace Shop;

public class DeliveryTest
{
    [Theory]
    [InlineData(4999, 490)]
    [InlineData(5000, 0)]
    [InlineData(1, 490)]
    [InlineData(25000, 0)]
    public void FeeFor(int total, int fee)
    {
        Assert.Equal(fee, Delivery.FeeFor(total));
    }

    [Fact(DisplayName = "an empty cart is refused")]
    public void EmptyCartIsRefused()
    {
        Assert.Throws<ArgumentException>(() => Delivery.FeeFor(0));
    }
}

Output

FAIL  FeeFor(total: 4999, fee: 490): Assert.Equal() Failure: Values differ Expected: 490 Actual: 0
PASS  FeeFor(total: 1, fee: 490)
PASS  FeeFor(total: 25000, fee: 0)
PASS  FeeFor(total: 5000, fee: 0)
PASS  an empty cart is refused
Tests: 5, passed: 4, failed: 1

Python

test_delivery.py

import pytest

from delivery import delivery_fee


@pytest.mark.parametrize("total, fee", [(4999, 490), (5000, 0), (1, 490), (25000, 0)])
def test_fee_for_total(total, fee):
    assert delivery_fee(total) == fee


def test_empty_cart_is_refused():
    with pytest.raises(ValueError):
        delivery_fee(0)

Output

FAIL  test_fee_for_total[4999-490]: assert 0 == 490
PASS  test_empty_cart_is_refused
PASS  test_fee_for_total[1-490]
PASS  test_fee_for_total[25000-0]
PASS  test_fee_for_total[5000-0]
Tests: 5, passed: 4, failed: 1

JavaScript

delivery.test.js

import { test } from "node:test";
import assert from "node:assert/strict";
import { deliveryFee } from "./delivery.js";

const cases = [[4999, 490], [5000, 0], [1, 490], [25000, 0]];
for (const [total, fee] of cases) {
  test(`deliveryFee(${total}) = ${fee}`, () => {
    assert.equal(deliveryFee(total), fee);
  });
}

test("an empty cart is refused", () => {
  assert.throws(() => deliveryFee(0));
});

Output

FAIL  deliveryFee(4999) = 490: 0 !== 490
PASS  an empty cart is refused
PASS  deliveryFee(1) = 490
PASS  deliveryFee(25000) = 0
PASS  deliveryFee(5000) = 0
Tests: 5, passed: 4, failed: 1

Exactly one case fails: 49.99 €. The name of the test tells you the amount, and the message tells you the expected and the actual value. You do not have to search for which step of which scenario.

More precisely, if you are curious

The output here is a short report, sorted by name: the tools really run, but inside a wrapper that prints only the line for each test and the count. The test names differ between languages because every tool builds them its own way: JUnit from the name of the parameterised test, xUnit from the method name and the arguments, pytest from the function name and the values, node:test from the text you give it.

The idea

Why not everything in Gherkin

Gherkin is for the conversation with the business: the examples agreed by the three amigos, written so that everyone can read them. As a testing tool it is expensive:

• every new line in a scenario needs a step, and steps have to be maintained;
• a scenario goes through more code, so it is slower;
• when it fails, the cause is further from the message.

And most importantly: fifteen technical scenarios hide the three that state the rule. Living documentation stays alive only while someone reads it.

So the scenarios are few and state the rule. The component tests are many and protect the code. This is the pyramid too: a wide base of fast tests, and above it fewer, slower ones that are closer to the customer.

More precisely, if you are curious

Aslak Hellesøy, who created Cucumber, has written more than once that Cucumber is first of all a tool for a shared language and collaboration, and that the tests are a by-product. If the scenarios are not written and read together with the business, what is left is an expensive way to write tests.

Exercise

The two levels

1. Which test protects what. The questions are about the scenarios from the previous lesson and the tests from this one.

1. You remove the 250.00 € case (25000 cents) from the parameterised test. Which bug will no test in the lesson catch any more?

The boundary with ">" instead of ">="The fee 4.99 € instead of 4.90 €A leftover from the old tariff: from 100 € delivery is 4.90 € again

2. Which test catches the bug "an empty cart is not refused but pays 4.90 €"?

Only the parameterised oneThe test "an empty cart is refused", and the scenario "An empty cart"None

3. The programmer wants to add 49.50 € both to the scenarios and to the component test. What do you say?

Only to the scenariosNowhere: 49.99 € is enough for everythingTo the component test yes; to the scenarios only if the business wants this example

Check

Test

Check yourself

1. What is the difference between a scenario and a component test for one rule?

The scenario is fasterThere is no differenceThe scenario is for the agreed examples and is read by the business; the component test calls the code directly and takes many technical cases

2. Why is a component test faster than a scenario?

It calls the method directly, without reading text, finding steps and going through the layers of the systemBecause it is in JavaIt is not faster

3. What does a parameterised test do?

It checks the parameters of the methodIt runs the same check with many sets of input data and expectationsIt runs the test several times with the same data

4. Why is not everything written in Gherkin?

Every scenario needs steps and maintenance, and many technical scenarios hide the rule from the businessBecause it is forbiddenBecause Gherkin cannot have numbers

Check

Start the lesson over

Related lessons

• Builds on: Test levels · Testing in the life cycle · Why and what to automate · From scenario to steps
• Continues in: API and user interface · Tests in CI · Promo codes: from the request to the release
• Tasks on this topic: Automation