I was speaking to a crew who were struggling with regression testing and after interrogating them they finally admitted that a large part of the reason they were struggling was that they did not really understand what they were testing.
I was shocked and horrified so I wandered off.
Soon after I spoke to some business analysts who were struggling with an agile project. They alleged that the agile approach they were using did not allow enough time to capture the requirements properly. But guess what, they broke down under interrogation and admitted that a large part of the problem they were having was that they did not really understand the product they were building. Apparently the evil agile people had forced them to write stories about the system before they had time to understand it.
So I started wondering – is their any way you could actually understand a system or product before you built and tested it. This sounds like something you would want to do even if evil agile people want you to go really fast. In fact the faster you need to go in building something the less time you probably have to spend on wasted effort that is not related to what you are building.
I have already described PAC as a simple approach to user analysis but that describes what the users do and not what the system does for them.
So here is an old school approach that won’t take long.
Start by looking at the project charter/mandate and then ask people what the system does. Then do a little exploratory testing if the system already exists.
Now you are ready for a quick one hour workshop (or one hour of sitting in a cafe if you are on your own).
Step 1 – book a room for an hour or go to a cafe and order a skim milk cappuccino
If you have some friends who also want to understand the product then bring them along. Otherwise sneak off to a cafe on your own. You will need a pen and paper and probably enough spare change to pay for you coffee.
Step 2 – create a one paragraph summary of what you think the product does
You can write you summary in any format you like, but I like the following approach which is loosely stolen from a book called “Stand back and deliver”. They use a similar approach to describe a team, but I find it works well enough for a product. It is also similar to the cereal box approach described in “Agile Project Management“.
The product provides [something] to [someone] so that they can [do something better]. Without it [someone] would have to [fall back]
If you are in a workshop setting, have each participant write this without speaking to the others. If you are on your own then I guess you should just write it out. But only allow 15 minutes for this exercise (plus 10 minutes to share and argue about the answers if in a workshop). You have a lot to do and you want to have time to drink our cappuccino.
For example, lets say that Jenny owns the coffee shop you are sitting in. She wants to set up a new “wine and coffee” club so that people can learn more about their favourite beverages, order premium beverages to consume on site and even order bulk purchases for home.
Jenny decides that she wants to set up a new database system to keep track of the members of the club and if she can get the budget she will also add a web page with a blog and a social community for club members.
What does Jenny’s new database do and who is it for? She could build it for the members or for herself … does it matter?
Jenny comes up with the following summary:
The coffee club database keeps track of coffee club membership information so that we can manage the coffee club. Without it we would need to have a manual system in place or forget the whole coffee club idea.
Optional step 2a – add some old school requirements (3-5)
With stories being all the rage these days, people sometimes often talk about the dark ages when we relied on requirements documents. But hidden in all those documents we could sometimes find some really clear and concise requirements that really did help us to build what people wanted.
First we used to list some functional requirements (what the system is meant to do) and then if you want to make good products you add some quality statements or “non-functional” requirements to clarify the style, performance and behaviour you want the system to have.
Here is my favourite format for functional requirements:
The [system or module] will [do something];
Or you can go all out and add two more two optional sections:
When [something] [the module] will [do something] if [some condition]
You can write millions of these one line requirements, but for the sake of understanding what the new product does you only really need about 3 – 5 lines. More than this will often cause people to start simply agreeing with them. Only listing the top 3 requirements will often cause people to have to think about what they mean.
Similarly, you can spend all day on this, but I would only allow 15 minutes, plus 10 minutes to share if you are in a workshop. Even 5 minutes might be enough.
So let’s see what Jenny came up with.
The coffee tracking system will store data on members of the coffee club
The coffee tracking system will store data on the coffees we have available and the promotions we are running at the moment with respect to those coffees
The coffee tracking database will store a history of promotions we have run and purchases users have made where privacy legislation and PCI compliance allow us to do so
The web page will provide static data on premium coffee and wine varieties and a blog for publishing regular coffee related news
When members join the online club the web page will provide access to a chat room where they can share coffee related views if the member has a valid login
Jenny has more work to do in order to fully scope and build the system. But she now has a one page description that she can use to check whether she and her IT vendor (Charles who works for free coffees) understand the system.
Optional step 2b – add a quality statement or measure
Even if a system does what we want, it might not do it the way we want it. For example
When we get home, my flatmate will cook dinner if I do the dishes after dinner
This sounds good – but I have had some flatmates who are great cooks and some who can make toast and coffee taste terrible.
So it is a good idea to define how you will know if the new system is good or not. Here is one (not too complicated approach):
The stakeholders will think [system or module] is great if [something)
The stakeholders will be really upset if [system or module does/does not … ] or [something happens]
Jenny might come up with:
I will be really annoyed if the system is hard for me and the barista to use
Customers will be upset if we leak their personal data or use it to spam them
I will be really happy if the system can run on my new galaxy tab as well as my netbook
Of course you are allowed to do better. A lot of people use iso9126 (a standard for software quality) to list the non-functional things the system should do (be reliable, be easy to learn etc). I often use FMEA (a standard way to be scared) or FAB (a sales technique I use for non-functional stuff).
Step 4 – Go and find out that you are wrong
You would think that it would be hard to get a one paragraph description of a system wrong. But on almost every project I have tried this on I have found that people disagree on it.
So the final step is to share this paragraph with the project sponsor, the product owner or the project owner. They will correct your understanding and often surprise you.
If you have this simple summary, I can almost guarantee you will be able to do a lot of things much more easily – requirements are easier and so too is regression testing for example.
The only downside is that it can make it harder for developers and product owners who want to build without having to think about what they are doing. And of course you might need to spend up to an hour in a coffee shop to do the hard part of the work.