Reviewing a specification
In short: A specification is read twice: once whole, then line by line with a checklist. Findings are in the text (words without a measure, ambiguities, contradictions) and outside it (cases the text says nothing about). Each one is recorded with its place, kind, severity and a suggestion. In the end each gets a decision: fix, question for the owner, or not a problem.
Problem
Two weeks before release
Promo codes go live in two weeks. The analyst has written the specification and sends it to you and two others for review. The deadline is tomorrow at 14:00, the discussion is on Friday.
You are a reviewer. You have the checklist from the previous lesson, the specification and one hour. The goal is clear: find as many problems as you can now, while each one costs a word.
The idea
How to read it
The specification is read twice.
First as a whole, without stopping. The question is "what is missing?". A rule that no sentence talks about is invisible when you read line by line.
Then line by line, with the checklist. For every sentence: who; is there a measure; what happens at the boundary and in the negative case; does it say what, not how; does it contradict anything.
Write things down right away, with the place. "P-3, "once": per customer or per order?" In an hour you will not remember which line it was.
Try it
The specification: the rules
The first part of the specification. Mark the problems and choose the kind of each.
Press the words that are a problem; press again to unmark. The arrow keys move from word to word.
The problems in the text
- “The user” no actor: Any visitor or a registered customer? Without an account the shop cannot know whether the same person has already used the code.
- “10 % discount” ambiguous: Of what: of the product prices or of the whole total with delivery? This is also where the open question sits: is the 50 € free delivery threshold checked before or after the discount? A question for the owner.
- “once” ambiguous: Once per customer or once per order? The shop's rule is "in one paid order", and the sentence has to say so.
- “until the end of the campaign” incomplete: When does the campaign end: date, time, time zone? The deadline has to be in the requirement itself or in the code's data, otherwise the tester has nothing to compare against.
- “in a pop-up window” solution instead of need: How the message is shown is a design decision. The requirement says what the customer sees.
- “an appropriate message” incomplete: Which message? The rule is "The code has expired". Without the exact text everyone writes their own, and the test has nothing to expect.
Fixed: "P-1. A registered customer enters a promo code in the cart. The code gives a 10 % discount on the products that are not discounted." (Whether it also applies to delivery remains a question for the owner.) "P-3. The customer can use each code in one paid order." "P-4. Each code has a last day of validity, until 23:59 Bulgarian time." "P-5. If the code has expired, the customer sees "The code has expired", and the total stays without a discount."
Try it
The specification: the discounts
The second part, and one requirement from the loyalty specification. Mark the problems and choose the kind of each.
Press the words that are a problem; press again to unmark. The arrow keys move from word to word.
The problems in the text
- “add up” contradicts: L-2 says the larger one applies. A loyal customer with a code buying a product that is not discounted gets 15 % by P-6 and 10 % by L-2. It is the argument from the lesson "When a table is clearer", now between two specifications.
- “the larger one applies” contradicts: It contradicts P-6. The shop's rule is "discounts do not add up, the larger one applies", so P-6 is fixed and L-2 stays.
- “quickly” unverifiable: Without a number there is no test. For example: "the answer about the code comes in under 1 second in 95 % of cases".
- “If possible,” unverifiable: A loophole: if the discount is not shown, one can always say "it was not possible". The customer has to see it before paying.
Fixed: "P-6. When both the code and loyalty apply to a product, the larger discount applies." "P-7. The answer whether the code is valid comes in under 1 second in 95 % of cases." "P-8. Before paying, the customer sees the discount as a separate line in the cart."
The idea
What is not there
The second reading finds the words. The first finds the gaps, and they are often more serious. In the promo code specification:
- The second attempt. P-3 says the code is used once, but what does the customer see when they try again? The rule is "The code has already been used". It is not in the text.
- The cancelled order. The order with the code is cancelled before it is shipped. Is the code used? The rule says "in one paid order". Whether it stays used after a cancellation, nobody has said. That is a question for the owner.
- The return. The customer returns a product bought with a discount. Are they refunded with the discount or without? Another question.
- Who creates the codes. Who creates them and who sets the deadline? Does accounting want to know how much discount was given?
None of these findings can be marked as a word. They are recorded separately, with the place "missing".
The idea
The review log
A finding that is not written down does not exist. The review log is a table the author fixes from without asking you.
| # | Place | Finding | Kind | Severity | Suggestion |
|---|---|---|---|---|---|
| 1 | P-6, L-2 | "add up" versus "the larger one applies" | contradicts | high | P-6: "the larger one applies" |
| 2 | P-1 | 10 % of what: the products, or delivery too? | ambiguous | high | question for the owner |
| 3 | P-3 | "once": per customer or per order? | ambiguous | high | "in one paid order" |
| 4 | missing | what the customer sees on a second attempt | incomplete | medium | "The code has already been used" |
| 5 | missing | is the code used after a cancelled order | incomplete | medium | question for the owner |
| 6 | P-5 | "in a pop-up window" | solution instead of need | low | remove it |
- The place says where. The requirement's number is better than "near the start".
- The kind says what is wrong, with the words from the lesson "The good requirement".
- The severity says how much it hurts if the defect reaches the customer. "The discounts add up" hurts with money on every order. "Pop-up window" hardly hurts.
- The suggestion is not an order. It shows you understood the problem and gives the author somewhere to start.
The log has no author's name. The document is reviewed.
More precisely, if you are curious
Severity in a review is the same idea as the severity of a defect, which you will meet in the module on the process: how serious the consequences are. Priority (when it will be fixed) is a separate decision. Teams often use a few levels: blocking, high, medium, low.
Inspections also collect metrics from the log: how many findings per page, of which kind, how much time went in. That way the team sees, for example, that "missing negative case" is the most common finding, and adds a question about it to the checklist.
Try it
The good record
The idea
The discussion and the decision
On Friday the findings of the three reviewers are put together. The moderator goes through them one by one. Each gets a decision:
- fix: the problem is clear, the rule is clear, the author fixes it;
- question for the owner: the rule does not exist, and nobody in the room can make it up;
- not a defect: the discussion shows the text is fine; the reason is recorded so it is not raised again.
In the end the document gets a decision too: accepted, accepted after the fixes, or needs a new review. With an open question about the price, work on the story cannot start. So the promo code specification goes back until the owner says what the 10 percent is calculated from.
Exercise
What happens to the finding
1. The decision. For each finding from the discussion, choose what happens to it.
Test