Equivalence partitions
In short: The values a rule treats the same way form an equivalence partition. 30 € and 40 € are one partition for delivery: if one total is right, so is the other. So one test per partition is enough. Coverage is how many of the partitions are tested. Invalid partitions are tested one at a time, so that one bug does not hide behind another.
Problem
Twenty tests, one idea
The tester gets the delivery rule and writes tests: a cart of 5 €, 10 €, 15 €, 20 € and so on up to 45 €. Then 55 €, 60 € and so on up to 100 €. Twenty tests, all of them pass.
After release a customer presses "Order" with an empty cart and gets an order of 0.00 € with a 4.90 € delivery fee.
The twenty tests checked two things: paid delivery and free delivery. Ten times each. The third thing, the empty cart, nobody checked.
The idea
Black-box techniques
Exhaustive testing is impossible. The possible cart totals run into millions. You need a way to pick few tests that say a lot. That is test design, and this module is about its techniques.
The first four are black-box techniques: the tests come from the rule, from the specification, not from the code. The tester sees the input and the output. What is inside is not needed.
| Technique | What it asks | Lesson |
|---|---|---|
| equivalence partitions | which values behave the same way? | this one |
| boundary values | where does one partition end and the next begin? | the next |
| decision table | what happens for every combination of conditions? | the third |
| state transitions | what happens along the path of an order? | the fourth |
Then come the white-box techniques (from the code) and the experience-based ones.
The idea
One partition, one test
The rule: "Delivery is free from 50.00 € inclusive, otherwise it is 4.90 €. An empty cart cannot be ordered."
It splits the totals into groups it treats the same way:
| Partition | Totals | Behaviour |
|---|---|---|
| empty cart | 0 € | refused |
| paid delivery | from 0.01 to 49.99 € | 4.90 € |
| free delivery | from 50.00 € up | 0.00 € |
Each group is an equivalence partition. If the system calculates 30 € correctly, it almost certainly calculates 40 € correctly too. So one test per partition is enough, a representative of the partition.
How many of the partitions are tested is the partition coverage. The twenty tests from the start had a coverage of 2 of 3. Three tests, one per partition, have 3 of 3.
Try it
Which is one partition?
The idea
The table of test cases
In the tasks below you write test cases as a table: an input and an expected result per row. "Run" runs them against two things:
- the correct system, where every row has to pass: if it does not, your expected result is wrong;
- versions of the shop with one bug each, the kind real shops have. A good set of tests fails on every one of them.
The answer says how many bugs you catch and where to look for the rest, not which row to add.
An empty cell in the expected result is not checked. Amounts are written as 49.99, and "Refused" is "yes" or "no".
Try it
The partitions of delivery
Write one test for each partition of the delivery rule. The fewer rows, the better.
The rule: Delivery is free for an order from 50 € inclusive, otherwise it is 4.90 €. An empty cart cannot be ordered.
| Input | Expected | The correct system | |||
|---|---|---|---|---|---|
| # | Cart total | Refused | Delivery | ||
An empty result cell is not checked. Amounts are written like 49.99.
Solution
| Cart total | Refused | Delivery |
|---|---|---|
| 0 | yes | |
| 30 | no | 4.90 |
| 80 | no | 0 |
Three rows, three partitions: 0 € (refused), 30 € (4.90 €), 80 € (0.00 €). The 30 € cart catches the version where the threshold is written as 5 € instead of 50 €. The 0 € cart catches the empty cart that pays delivery. And 40 € next to 30 € catches nothing new: the two rows are one partition.
The idea
Valid and invalid partitions
Partitions come in two kinds. Valid ones are the values the rule processes. Invalid ones are those the system has to refuse: a negative total, a negative number of items, text instead of a number.
When there are several inputs, each has its own partitions. The cart limit ("at most 10 of one product, you add at least 1 item") has two inputs: how many are in the cart and how many are added. The partitions of the result are three:
- the adding is accepted: 2 in the cart, I add 3, it becomes 5;
- refused because it would be more than 10: 5 in the cart, I add 8;
- refused because the quantity is invalid: I add -2 or 0.
And one rule for invalid partitions: one per test. If a row has both -2 items and 15 in the cart, the system will refuse it. But because of which? If the check for a negative number is missing, the test still passes, because the limit saved the day.
More precisely, if you are curious
ISTQB CTFL v4.0.1 describes the technique in section 4.2.1 (equivalence partitioning). Coverage is measured as the number of tested partitions over all partitions, as a percentage. With several inputs, the simplest measure is Each Choice coverage: every partition of every input is tested at least once. It does not require all combinations between the inputs. Combinations are the topic of decision tables and of pairwise testing.
Partitions are not only for inputs. Outputs are split too (accepted, refused), and times (before the deadline, after it), and anything the rule treats differently.
Exercise
The partitions of the limit
1. The cart limit. Here there are two inputs, and the partitions follow the result.
Write one test for each partition: accepted, refused because of the limit, refused because of an invalid quantity. Check what stays in the cart too.
The rule: The cart can hold at most 10 of one product. Adding beyond 10 is refused with a message, and the cart stays the same. You add at least 1 item.
| Input | Expected | The correct system | ||||
|---|---|---|---|---|---|---|
| # | In the cart | Adding | Refused | In the cart after | ||
An empty result cell is not checked. Amounts are written like 49.99.
Solution
| In the cart | Adding | Refused | In the cart after |
|---|---|---|---|
| 2 | 3 | no | 5 |
| 5 | 8 | yes | 5 |
| 3 | -2 | yes | 3 |
Three rows: (2, add 3) is accepted and becomes 5; (5, add 8) is refused, and the cart stays at 5; (3, add -2) is refused, and the cart stays at 3. If all rows start with an empty cart, two bugs slip through: the version that compares only the added items with 10, and the one that empties the cart on a refusal. With an empty cart both look correct.
The idea
What the black box does not see
Partitions come from the rule. And the rule does not say everything about the code.
Say a line from the old tariff has stayed in the code: "from 100 € delivery is 4.90 € again". For the rule, 80 € and 150 € are one partition. For the code they are two. The test with 80 € will pass, and the bug will stay.
The black box cannot see a partition that exists only in the code. That is why there are other techniques: the white box looks at the code, and experience suggests where such things usually hide. And the boundary between partitions, where bugs are densest, is the next lesson.
Test