Skip to content
wannacode

Living documentation

In short: A Word document goes out of date quietly. Scenarios run on every change and fail as soon as the system stops doing what they say: that is living documentation. They are written before the work, by the three amigos: business, developer and QA. BDD is that conversation. Scenarios written by QA alone after the code are just tests with more work.

Problem

The document that lies quietly

A new person on the team opens the delivery specification. A Word document from two years ago. It says: "free delivery over 40 €".

The shop gives free delivery from 50 €. Someone changed the rule, but not the document. Nobody noticed, because the document does nothing: it just sits there.

Now the new person does not know which one is right: the document or the system?

The idea

Documentation that runs

The scenarios in this module are both a document and a test. If the rule changes in the system but not in the scenario, the scenario fails on the next run. If it changes in the scenario but not in the system, it also fails. The two cannot drift apart quietly.

This is called living documentation. It is alive on one condition: the scenarios run on every change, usually automatically. Scenarios that nobody runs go out of date just like the Word document.

To be documentation, the file must also read like it:

  • Feature says what the file describes, and the story itself can stand under it;
  • Rule is each business rule, in the owner's words;
  • under each rule are the examples: scenarios and outlines with concrete numbers.
Feature: Free delivery
  As a customer
  I want free delivery on a bigger order
  so that I do not pay 4.90 € for delivery.


  Rule: An empty cart cannot be ordered


    Example: A cart with no products
      Given an empty cart
      When I place the order
      Then the order is refused

The text under "Feature" is not a step. It is a description: Gherkin skips it when running, and people read it.

The idea

Who writes them and when

Scenarios are not written by one person alone. They come out of a conversation between three people you already met at Example Mapping. They are called the three amigos:

  • the business (the owner or the analyst) states the rules and what matters;
  • the developer asks how it will be built and what in it is hard;
  • QA asks "what if…?": the boundaries, the empty cart, the refusal.

And when? Before the work. A scenario written before the code says what is wanted. A scenario written after the code describes what was built, bugs included.

The Cucumber documentation calls the three steps discovery (a conversation about concrete examples), formulation (the examples are written down so that they can run, and the business confirms them) and automation (the developer binds the steps to the system). The third step has its own module later.

Try it

From the story to the document

Put the path from the story to living documentation in order.

  1. The developer binds the steps and builds the feature
  2. The owner brings the story
  3. The scenarios pass and the story is done
  4. The three amigos lay out rules, examples and questions
  5. The business reads the scenarios and says "yes, that is it"
  6. On every later change the scenarios run again
  7. The examples are written down as Gherkin scenarios

The idea

BDD versus "Gherkin tests"

All this has a name: BDD (behaviour-driven development). The important part of BDD is not Gherkin. It is this order: conversation, examples, then code.

Many teams take only the syntax. QA writes the scenarios alone, after the code, with "click the button". The business never reads them. These are Gherkin tests: ordinary automated tests with one more layer on top that has to be maintained. The cost of BDD is paid, and the benefit is missing.

Signs that this is happening:

  • nobody from the business has read the scenarios;
  • the scenarios are written after the code;
  • the steps are about buttons and fields, not rules;
  • scenarios get fixed when they fail, without asking whether the system got it wrong.

If there is no conversation, the honest choice is a plain component test. It is faster and cheaper. Gherkin pays off when the business really reads the scenarios.

Try it

BDD or not?

1. QA writes the scenarios alone, after the developer has finished. Nobody from the business reads them.
2. The owner, the developer and QA discuss the examples before the work. The scenarios come out of the session and the owner confirms them.
3. The team writes scenarios before the code, but all of them are "click the button" and "open the page".
4. The scenarios are run only by hand, once a year.

Exercise

The delivery document

The whole feature. The story and the rules are written.

Under each rule, put the examples that show it: around the boundary, far from it and the empty cart. The document must catch all six bugs.

GherkinFree delivery
More precisely, if you are curious

ISTQB CTFL v4.0.1. Section 2.1.3 ("Testing as a Driver for Software Development") puts TDD, ATDD and BDD side by side: all three define the tests before the code and apply the principle of early testing. For BDD it says the test cases are written in a simple form of natural language that stakeholders understand, usually Given/When/Then, and are then turned into automated tests. Section 4.5.3 describes ATDD (acceptance test-driven development): a specification workshop where the story and its criteria are discussed together (customer, developer, tester), then the test cases, first the positive ones, then the negative ones, then the non-functional ones. When they are in a format a test automation framework understands, the acceptance tests become executable requirements.

ISTQB CT-AcT v1.0 (acceptance testing) has a chapter on collaborative acceptance testing and treats acceptance criteria and acceptance tests as the shared work of business analysis and testing.

Dan North describes BDD in the article "Introducing BDD", first published in Better Software magazine in March 2006. In it he and Chris Matts create a template for acceptance criteria as scenarios: Given (the context), When (the event), Then (the outcomes).

The three amigos. The name is widely used in the agile community. It is often linked to George Dinwiddie, but we did not find a primary source that confirms it.

The Cucumber documentation ("Behaviour-Driven Development") describes the three practices: Discovery, Formulation and Automation.

Test

Check yourself

1. What makes scenarios "living" documentation?
2. Who are the three amigos?
3. When are scenarios written in BDD?
4. What is ATDD according to ISTQB CTFL v4.0.1?

Made by Ivo Gergov as part of Evo Tech's campaign for accessible programming education ·