State transition testing

Problem

The paid cancelled order

A customer cancels a new order. Then, in another browser tab where the payment page is still open, they press "Pay". The payment goes through. The order becomes "paid", the warehouse packs it, the courier delivers it.

The order tests were passing: paying for a new order, cancelling a new order, shipping a paid one. Each of them was an allowed transition. Nobody had tried a forbidden one.

The idea

The machine as tests

In the lesson "When a table is clearer" the state machine was a requirement. Here it is in full, with the transition that was missing there:

From	Event and guard	To
new	the customer pays	paid
new	the customer cancels	cancelled
paid	the customer cancels (before shipping)	cancelled
paid	the warehouse hands over the parcel	shipped
shipped	the courier delivers	delivered
delivered	the customer requests a return, within 14 days of receipt	returned

Six states, six valid transitions. "Cancelled" and "returned" are final: there is no way out of them.

A test here is not one input and one output. It is a path: a sequence of events, and after each one the state is checked. This is state transition testing.

In the tasks of this lesson a row of the table is a path from a new order ("pay, ship"), and what is checked is the state at the end and whether the last event was refused. Next to the table is a diagram of the machine: go to a row and it lights up along its path.

The idea

Three measures

How many paths are needed? It depends on what you want to cover.

All states: every state is visited at least once. Here 2 tests are enough. Which ones, you will find in the task.
All valid transitions: every row of the table is taken at least once. The two tests take 5 of the 6 transitions, and the sixth needs a third. 3 tests.
All transitions: the valid ones, and every invalid one, attempted and refused. Here the tests become many more.

Each measure includes the previous one. All states is the weak one: the bug from the start lived in a transition no path reached.

More precisely, if you are curious

ISTQB CTFL v4.0.1 describes the technique in section 4.2.4 (state transition testing) with these three measures: all states coverage, valid transitions coverage (also called 0-switch coverage) and all transitions coverage. The machine can be written as a diagram or as a state table. The table shows the invalid transitions too, the diagram usually does not.

For critical systems (trains, medical devices) tests are not enough. There the state machine is also checked formally: a tool proves that a forbidden state cannot be reached by any path. That is model checking, a kind of formal verification. IEEE 1012 ties the amount of checking to the integrity level: the more expensive the failure, the more checking. For the shop formal checking is too expensive, for a train's brake it is not.

Try it

All states

Write the paths that visit each of the six states. Two rows are enough.

The rule: A new order is paid, a paid one is shipped, a shipped one is delivered, a delivered one is returned. A new or paid order can be cancelled before it is shipped. Cancelled and returned are final. Any other event is refused, and the order stays where it is.

Go to a row and the diagram lights up its path.

	Input	Expected		The correct system
#	Path	Ends as	Last one refused	The correct system

Row 1

A path is a list of events from a new order: “pay, ship” (with a comma or a space). Write the state as a word: new, paid, shipped, delivered, cancelled, returned. An empty result cell is not checked.

The idea

Invalid transitions

The transition table lists what is allowed. More useful for testing is the state × event table: every cell says what happens if the event arrives in that state.

State	pay	cancel	ship	deliver	return
new	paid	cancelled	—	—	—
paid	—	cancelled	shipped	—	—
shipped	—	—	—	delivered	—
delivered	—	—	—	—	returned (within 14 days)
cancelled	—	—	—	—	—
returned	—	—	—	—	—

30 cells, 6 valid. The other 24 are invalid transitions ("—"). Each is a test: try it and check it is refused.

A test of an invalid transition checks three things: the attempt is refused, the state has not changed, and the customer sees why. And one invalid attempt per test: if the first one wrongly goes through, the order is already in another state, and the second attempt tests something else.

Try it

Is it forbidden?

1. The order is shipped. The customer presses "Cancel". What does the test expect?The order becomes newThe cancellation is refused, the order stays shipped, the customer sees how to return the productThe order becomes cancelled

2. Why are invalid transitions tested one per test?After the first refusal the order is not where the test thinks, and the next steps check nothingBecause there are few of themIt is not needed, they can all go in one test

3. A delivered order, the customer requests a return on day 16. What is the test?It is not a state testA valid transition to returnedAn attempted transition whose guard is not met: the return is refused, the order stays delivered

The idea

Guards on transitions

Some transitions have a guard. "Delivered → returned" happens only if the return is requested within 14 days of receipt. The event is one, but the result depends on the guard.

This is where the techniques meet. The guard is a number, so it has a boundary: day 14 is accepted, day 15 is refused (if the parcel was received on the 1st, that is the 15th and the 16th). The machine says where to test, boundary values say with what.

The same goes for "paid → cancelled": the guard is "before shipping". The test is two: cancelling a paid order before the warehouse hands it over, and an attempt to cancel right after.

Exercise

All transitions

1. All transitions and the invalid ones. The invalid event goes last in the path: that way the row checks only it. Start with the riskiest: paying, shipping, cancelling or returning when you should not. The "Last one refused" column matters: for some bugs the state is the same, and the only difference is the refusal.

Write paths for each of the six valid transitions and one row for invalid transitions until you catch every bug.

The rule: A new order is paid, a paid one is shipped, a shipped one is delivered, a delivered one is returned. A new or paid order can be cancelled before it is shipped. Cancelled and returned are final. Any other event is refused, and the order stays where it is.

Go to a row and the diagram lights up its path.

	Input	Expected		The correct system
#	Path	Ends as	Last one refused	The correct system

Row 1

A path is a list of events from a new order: “pay, ship” (with a comma or a space). Write the state as a word: new, paid, shipped, delivered, cancelled, returned. An empty result cell is not checked.

Solution

Path	Ends as	Last one refused
pay, ship, deliver, return	returned	no
cancel	cancelled	no
pay, cancel	cancelled	no
pay, ship, cancel	shipped	yes
pay, ship, return	shipped	yes
pay, pay	paid	yes
cancel, pay	cancelled	yes
cancel, ship	cancelled	yes
ship	new	yes

Test

Check yourself

1. The tests visit all states. Do they also take all transitions?Yes, alwaysNot necessarily: paid → cancelled may stay uncheckedYes, if there are at least three tests

2. Which measure is the strongest?All statesAll valid transitionsAll transitions, both valid and invalid

3. The "state × event" table for the order has 6 states and 5 events. How many cells are invalid transitions?63024

4. What is a guard on a transition?Something that has to be true for the transition to happen on the event, for example "within 14 days of receipt"A state with no transitionsAn event that happens only once

5. Why are invalid transitions the most forgotten tests?Because there are fewer of them than valid onesBecause they cannot be testedBecause the description of behaviour says what happens, not what must not happen

State transitions

Cookies