What quality is
In short: Quality is how far a product gives the people around it what they need. ISO/IEC 25010:2023 splits it into nine characteristics: one for what the system does and eight for how well it does it.
Problem
Everything works. Customers leave
After the new shop goes live, all 300 tests pass. Delivery is computed right, promo codes work, cancelling an order is allowed only before shipping.
But the cart page takes 9 seconds to load on a phone. Half the customers close it before they see the total. The program does everything right. And it is still bad.
So "right" is not the whole of quality. What is the rest?
The idea
Quality for whom?
One and the same shop is "good" for different reasons to different people:
- the customer wants to find the product, pay quickly and not have their card stolen;
- the warehouse wants the order to reach its software in the right format;
- accounting wants totals and discounts to add up to the cent;
- the developer who will change the delivery threshold next year wants to find it in one place.
Quality is how far the product gives these people what they need: both what is said and what is not. So the first question is not "is it good" but "for whom, and at what".
The idea
The nine characteristics
The ISO/IEC 25010:2023 standard arranges product quality into nine characteristics. That is a quality model: a list so that you forget nothing.
| Characteristic | In the shop |
|---|---|
| Functional suitability | Delivery is free from 50 €, the discount is taken off correctly. |
| Performance efficiency | The cart opens in under 2 seconds with 500 customers at once. |
| Compatibility | The order reaches the warehouse software and the courier in their format. |
| Interaction capability | A customer with a screen reader orders on their own; an error says what to do. |
| Reliability | If the courier does not answer, the order is kept and sent later. |
| Security | Nobody sees other people's orders; a promo code cannot be guessed. |
| Maintainability | The 50 € threshold is changed in one place, and the tests say what is affected. |
| Flexibility | The shop takes five times more customers on Black Friday and opens in a new country. |
| Safety | Food without listed allergens is not sold: the system holds it back instead of showing it. |
More precisely, if you are curious
ISO/IEC 25010 defines product quality as the degree to which a product satisfies the stated and implied needs of its stakeholders. The 2023 edition changed a few things from 2011:
- safety is a new, ninth characteristic;
- usability is now called interaction capability and has new subcharacteristics, for example inclusivity and self-descriptiveness;
- portability is now called flexibility and includes scalability.
Each characteristic has subcharacteristics. Functional suitability, for instance, is completeness, correctness and appropriateness. Since 2023 the quality-in-use model (how the person using the product fares) is in a separate standard, ISO/IEC 25019.
Exercise
Sort the complaints
1. Customer complaints. Every complaint is real. Which characteristic does it belong to?
The idea
Functional and non-functional
The first of the nine characteristics says what the system does: for a cart of 50 €, delivery is 0 €. Those are the functional requirements.
The other eight say how well it does it: how fast, for whom, how securely, how long without interruption, how quickly it can be changed. Those are the non-functional requirements.
Non-functional ones are forgotten most often. Nobody says them, because they "go without saying": of course the shop must be fast and secure. Until it turns out that it is not. In the opening story all the functional tests passed. One non-functional requirement was missing.
Try it
What, or how well?
The idea
Quality is agreed
"The site should be fast" cannot be checked. How fast? Which page? With how many customers? On what phone?
The sentence becomes a requirement when it gets a measure: "95% of cart openings take under 2 seconds with 500 customers at once, on a mobile network". Now the tester knows what to measure, and the owner knows what was promised.
The characteristics pull against each other. Logging in with an SMS code makes the shop more secure and slower for the customer. So not all of them weigh the same, and how much each weighs is a business decision. In the shop the speed and security of payment weigh more than safety. In hospital software it is the other way round. That is the principle "testing is context dependent", applied to requirements.
The idea
Verifying and validating quality
The two questions from the first lesson apply here too.
- Verification: the requirement says "under 2 seconds", and the measurement is 1.8 seconds. It passes.
- Validation: but is 2 seconds the right limit? Was it measured where the customers are: on a phone, on a mobile network? If customers still give up, the requirement is wrong, not the program.
A non-functional requirement can have a defect like any other. And the same thing catches it: a question and a concrete example, before any code is written.
Test