But where do you start if you want to do effective regression testing, but you are already busy and don’t want it to be huge burden?

I hope this article and a couple that follow will turn out to be a guide to one painless way to get regression testing up and running. Let me know if it helps.

Day one – first thing in the morning

Start by getting some context before you expend a lot of energy testing what you don’t understand. I have put some tips in this article, but you can also just do the obvious things:

• Read the project charter if there is one, otherwise sit down and create one
• Sit down and discuss the users of the system you are building and the product you are building
• Document what you found, even if it is just on the whiteboard or on napkins

This can all be done in under one hour and it is worth getting the regression testing team to sit down together to do it. Of course you can feel free to take 2 hours if you want to fully understand what you are testing

Day one – 20 minutes of scripting

Create some scenarios that you will use for testing.  This does not mean that you have to fully document well written test scenarios, since that comes later.

The easiest way is to simple use the PAC table you created and come up with some examples of what some users might do.

I am using the example project from a previous blog where a cafe owner wants to :

This is our initial map for our regression testing. It won’t last long but it gives us a place to start.

Spend an hour or two doing exploratory testing

Now that you have some rough scenarios, log into a test version of the system and try to run some tests.

The tests you run will be “exploratory” because you have not created any instructions yet. But for each scenario, don’t just log in and make things work, instead try this:

• Try to run the scenario successfully from end to end and then record the outcome
  • Write any bugs or “strange things” on a piece of paper or whiteboard
  • Write down how long it took to run the scenario
• Stop and ponder for a moment, Use the international standard for being scared to come up with things that might go wrong. Then try to run the scenario again seeing if it will fail.
• As you run the scenario a second or third time, try using the international standard for being stupid to come up with mistakes or “strange things” the users might do at each step during the scenario.
• Document your findings again
  • Scenario number
  • Potential errors or questions I found
  • Rough steps I followed

Go and report the potential errors you found and if they turn out to be errors then record them in whatever bug tracking system the project uses.

Discuss the context again for 20 minutes

Sit down with the other people doing regression testing and discuss how things went

Provide the following “management report”:

Tests pass if you found no errors that went into the bug tracking system. They fail if you did record errors. Passing or failing is not good or bad, it just shows which scenarios found something.

Now sit down with the group again and discuss the context your are testing in

• What led to errors?
• What did you learn?
• Should you change the PAC or Product statement? Should you change the scenarios you are running.

There is more to do tomorrow, but you have already done more thorough regression testing than a lot of teams do. Well done.

Then I worked on a project with a mining company and one of the consultants explained human factor analysis to me in simple terms. He told me that mine sites can be dangerous and part of his job was to “stop people killing themselves when they are stupid”.

I suggested he stop hiring stupid people but he told me that they tried that and it didn’t work. Apparently you can be really intelligent on a mine site 800 days in a row but then be stupid for 10 minutes one day and be in an accident and then be killed.

“Luckily we have a standard for being stupid though” he said

I was surprised that he could get people to comply with a standard for making mistakes, but then he showed me “iso90210 (not a real standard) – the standard mistakes you can make when using a machine”.

1. Action omitted
2. Action too early
3. Action too late
4. Action too much/too often
5. Action too little/not often enough
6. Action too long
7. Action too short
8. Action in wrong direction
9. Right action on wrong object
10. Wrong action on right object
11. Wrong action on wrong object
12. Information not obtained/transmitted
13. Wrong information obtained/transmitted

It seemed like a long list of obvious things, which is what it turned out to be. I found when I was analysing processes (which I was doing at the time) that many errors could occur in the process when one of the things happened in the list. So I combined this with the “international standard for being scared” (FMEA) to find I could tear most processes apart and find errors really quickly.

Even if you can’t break the process instantly (which you usually can), you can almost always break it if you:

• Apply one standard mistake; and
• See what happens and then apply a second one rather than simply moving on.

This also applies to computer applications. For example

• The user gets to question 5 and realises they do not have the information they need to complete the form.
• The system gives them a polite error message.
  • Rather than completing the form or hitting cancel (the right thing to do) they exit by hitting the “back button” or closing the browser (right action wrong object); or
  • They hit cancel and nothing happens so they hit it again 3 times (too often) and third time they hit the button the browser decides to reload and process the click on the wrong button (something from the next screen).

I find that even though I am not a tester, I can often break applications in regression testing by assuming my users will comply with the standard for being stupid at least once or twice.

You can take things a lot further if you study “human factors” because it turns out that “human factor analysts” (I am not sure what they call themselves) have spent a lot of time trying to find common reasons for all kinds of things. For example:

• Common reasons why people do the wrong thing deliberately (peer pressure, not enough time in the day, rewarded for wrong behaviour etc),
• Common reasons people make mistakes (information overload, falling back on how the old system worked, lack of knowledge, lack of caring etc).

So there seems to be a whole field of study about how to make mistakes properly. I guess if you are going to do something you may as well do it properly … and since we make mistakes so often we may as well get really good a making them

But what is of more use to IT projects where people want to build good systems:

• If your system could kill people, probably get an expert in safety to test it.
• Even if your system won’t kill people … surely you want to anticipate the most obvious mistakes that users will almost certainly make.  So it follows that you would also want to at least do some simple testing to find out what happens when they do.

So I think it is fair to say that any team who want to produce good processes, good IT applications or good products will do some regular testing to see what happens when “the dumb users” make the same predictable mistakes we all tend to make.

So I would like assume every developer, BA and tester would do some simple exploratory testing  as they add features and I would assume that every team would do some regular regression testing as their system emerges.

But maybe that is the uncharted territory for human factor analysts – why do IT teams consistently make the same mistake of assuming the users will do exactly what they are meant to, when everybody knows they won’t?

Why would anyone spend$5m and 6 months of their life on a project and then only test how a user might actually use the system at the  very end of the project when there is no time left to modify the way the system behaves?

When you think about it though, shouldn’t we do a similar thing to define our project purpose when we start the project. I guess the answer is yes, but it could turn into a lot of work. So what we need is an easy, lazy approach.

This is what I think you can do on just about any project.

1. Define the problem you are meant to solve.
2. Define your scope.
3. Flavour to taste:
   1. Consider PAC to define the users
   2. Consider defining your product
   3. Consider adding an estimate
   4. Consider listing risks and other bad stuff

Paragraph one – choose one of these problem statements

• Problem statement from the book Fierce Conversations by Susan Scott (you should really read this book if you want to be both lazy and successful at the same time):
  • The problem we have is ….
  • It matters because ….
  • Ideally what we would like is …
  • So far we have done …..
  • Therefore, what we want to achieve next is ……
• Focussing question
  • How can [We] provide [Something] to [someone] so they can [do something or get some benefit or deal with some issue]
  • Eg – how can the IT team provide a database to Jenny so she can run her coffee club. OR how can Jenny provide a coffee club to her customers so they can buy coffee to take home
• The old fashioned story/legend format, which is the simplest you can use.
  • Currently [things are like this] but it sucks because [some annoyance] so we will [resolve the issue]
  • For example Jenny runs a coffee shop and she would like to sell take home coffee to customers so we want to create a coffee club to do this. Or Jenny has set up a coffee club for her customers but she is losing track of all the information she needs to track so we are gong to build her a database.

You can find a couple of other problem statements in a previous article.

Paragraph two – say what is in scope and what is not

No project ever fully solves a problem from every possible angle. If you build software, maybe you should train people and maintain the software. But even if you maintain the software for 5 years, at some point someone should replace the software with a better solution. But even if your project team agree to come back and replace the software, will they be the ones who pay for the carbon offset to make up for the extra greenhouse gases produced during the project lifecycle.

People generally understand that some things will be in scope and that others will be out of scope. Yet we often make wildly different assumptions about what actually goes into the scope.

So I think it is useful to define the following in at least some detail:

• This is in scope – deploying a database, maintaining it in production; and
• This is out of scope – ongoing training, maintaining the customer data.

This is often put into a table with the in-scope stuff on the left and the out-of-scope on the right.

But if you are a vendor or you are delivering to a demanding internal client you might choose to go one step further:

• The project team will deliver (in scope) – deploying a database
• The customer will – provide the team with access to users, supply coffee, provide testing
• The following is out of the scope of this project – ongoing training

Paragraph three onwards.

• Add constraints or priorities if you want to.  For example it needs to be in production before Friday next week (eek)
• Add PAC, Product statement or risks if you like
• Add an estimate of the effort involved if you want to
• Add the name of the sponsor, the provider and others if you really want to push the content.

If you want to you can even add a communication plan, a view of wether users will actually adopt the change we are implementing or a test strategy, but not all projects will need this much detail at the beginning.

A simple estimate

People often hesitate to provide estimates. But you can provide something as simple as this

We are 90% confident that the project will take between X and Y.  We could be more confident if [something]. For example:

• We are 90% confident the project will take between 50 and 150 years (let’s kill it now);
• We are 90% confident the project will take 3 developers between 3 and 5 days (let’s do it)’; or
• We are 90% confident the project will take between 3 weeks and 4 months. Hmm -
  • We can kill it if 3 weeks is too long;
  • We can get started if 4 months is tolerable and report back as we go. We will be more confident once we install the invergolator and get it running on windows 7; or
  • We could do more analysis.  This would add one week to the project but when we understand your needs better and review the deployment options for invergolators then we can provide a better estimate.  Let me know what you want us to do.

I have often found these estimates are enough to make informed decisions. But if you want to get far better at estimating then you should probably read “How to measure anything”.  That book even teaches you to estimate and understand the value of doing more estimates.  But the lazy approach is just to do what I did above.

What do you think?

It seems pretty basic doesn’t it.  So your project should probably have a more detailed description that everyone understands (the stakeholders and the team working on the project). But in practice I have often found a simple summary of the project is enough to make do.

In fact my quote for the day is :

Projects where everyone has the same simple understanding of the problem do better than those where some of the team have a really detailed understanding that is different to what the rest of the team understand.

I was shocked and horrified so I wandered off.

Soon after I spoke to some business analysts who were struggling with an agile project. They alleged that the agile approach they were using did not allow enough time to capture the requirements properly. But guess what, they broke down under interrogation and admitted that a large part of the problem they were having was that they did not really understand the product they were building.  Apparently the evil agile people had forced them to write stories about the system before they had time to understand it.

So I started wondering – is their any way you could actually understand a system or product before you built and tested it. This sounds like something you would want to do even if evil agile people want you to go really fast. In fact the faster you need to go in building something the less time you probably have to spend on wasted effort that is not related to what you are building.

I have already described PAC as a simple approach to user analysis but that describes what the users do and not what the system does for them.

So here is an old school approach that won’t take long.

Start by looking at the project charter/mandate and then ask people what the system does. Then do a little exploratory testing if the system already exists.

Now you are ready for a quick one hour workshop (or one hour of sitting in a cafe if you are on your own).

Step 1 – book a room for an hour or go to a cafe and order a skim milk cappuccino

If you have some friends who also want to understand the product then bring them along. Otherwise sneak off to a cafe on your own.  You will need a pen and paper and probably enough spare change to pay for you coffee.

Step 2 – create a one paragraph summary of what you think the product does

You can write you summary in any format you like, but I like the following approach which is loosely stolen from a book called “Stand back and deliver”. They use a similar approach to describe a team, but I find it works well enough for a product. It is also similar to the cereal box approach described in “Agile Project Management“.

The product provides [something] to [someone] so that they can [do something better]. Without it [someone] would have to [fall back]

If you are in a workshop setting, have each participant write this without speaking to the others. If you are on your own then I guess you should just write it out. But only allow 15 minutes for this exercise (plus 10 minutes to share and argue about the answers if in a workshop). You have a lot to do and you want to have time to drink our cappuccino.

Example

For example, lets say that Jenny owns the coffee shop you are sitting in. She wants to set up a new “wine and coffee” club so that people can learn more about their favourite beverages, order premium beverages to consume on site and even order bulk purchases for home.

Jenny decides that she wants to set up a new database system to keep track of the members of the club and if she can get the budget she will also add a web page with a blog and a social community for club members.

What does Jenny’s new database do and who is it for?  She could build it for the members or for herself … does it matter?

Jenny comes up with the following summary:

The coffee club database keeps track of coffee club membership information so that we can manage the coffee club. Without it we would need to have a manual system in place or forget the whole coffee club idea.

Optional step 2a – add some old school requirements (3-5)

With stories being all the rage these days, people sometimes often talk about the dark ages when we relied on requirements documents. But hidden in all those documents we could sometimes find some really clear and concise requirements that really did help us to build what people wanted.

First we used to list some functional requirements (what the system is meant to do) and then if you want to make good products you add some quality statements or “non-functional” requirements to clarify the style, performance and behaviour you want the system to have.

Here is my favourite format for functional requirements:

The [system or module] will [do something];

Or you can go all out and add two more two optional sections:

When [something] [the module] will [do something] if [some condition]

You can write millions of these one line requirements, but for the sake of understanding what the new product does you only really need about 3 – 5 lines. More than this will often cause people to start simply agreeing with them. Only listing the top 3 requirements will often cause people to have to think about what they mean.

Similarly, you can spend all day on this, but I would only allow 15 minutes, plus 10 minutes to share if you are in a workshop. Even 5 minutes might be enough.

So let’s see what Jenny came up with.

The coffee tracking system will store data on members of the coffee club

The coffee tracking system will store data on the coffees we have available and the promotions we are running at the moment with respect to those coffees

The coffee tracking database will store a history of promotions we have run and purchases users have made where privacy legislation and PCI compliance allow us to do so

The web page will provide static data on premium coffee and wine varieties and a blog for publishing regular coffee related news

When members join the online club the web page will provide access to a chat room where they can share coffee related views if the member has a valid login

Jenny has more work to do in order to fully scope and build the system.  But she now has a one page description that she can use to check whether she and her IT vendor (Charles who works for free coffees) understand the system.

Optional step 2b – add a quality statement or measure

Even if a system does what we want, it might not do it the way we want it.  For example

When we get home, my flatmate will cook dinner if I do the dishes after dinner

This sounds good – but I have had some flatmates who are great cooks and some who can make toast and coffee taste terrible.

So it is a good idea to define how you will know if the new system is good or not. Here is one (not too complicated approach):

The stakeholders will think [system or module] is great if [something)

The stakeholders will be really upset if [system or module does/does not … ] or [something happens]

Jenny might come up with:

I will be really annoyed if the system is hard for me and the barista to use

Customers will be upset if we leak their personal data or use it to spam them

I will be really happy if the system can run on my new galaxy tab as well as my netbook

Of course you are allowed to do better. A lot of people use iso9126 (a standard for software quality) to list the non-functional things the system should do (be reliable, be easy to learn etc). I often use FMEA (a standard way to be scared) or FAB (a sales technique I use for non-functional stuff).

Step 4 – Go and find out that you are wrong

You would think that it would be hard to get a one paragraph description of a system wrong.  But on almost every project I have tried this on I have found that people disagree on it.

So the final step is to share this paragraph with the project sponsor, the product owner or the project owner. They will correct your understanding and often surprise you.

If you have this simple summary, I can almost guarantee you will be able to do a lot of things much more easily – requirements are easier and so too is regression testing for example.

The only downside is that it can make it harder for developers and product owners who want to build without having to think about what they are doing. And of course you might need to spend up to an hour in a coffee shop to do the hard part of the work.

Regression testing is simply the idea that when you make changes to a system you should test to make sure you haven’t broken what was already there. But for reasons unknown to psychologists, IT teams fall into a strange psychological pattern whenever asked to do regression testing.  This is the opportunity for someone to do a Phd – we understand that people consistently adopt the same four stage cycle of pain, but social scientists are at a loss to explain why:

1. Regression test denial.  People on most projects swear that there is no time for regression testing at the moment.
   • They will admit that in the last project they spent an inordinate amount of time (and frustration) near the end of the project trying to understand and fix problems that were created early in the project. They will also observe that most of these problems came to light as soon as regression testing finally started.
   • They will also complain about “the business” being indecisive and asking for new things every showcase (demonstration of the software).
   • The business will say their “alleged new change” was an obvious part of the requirements but the team will say they could not have anticipated it.
   • A lot of time will be spent fixing these new demands, but even more time will be spent arguing about them and demanding ever greater detail in the requirements, even though we still keep missing things and the business keep changing their mind. 
   • I accept that better requirements will address some of these issues and better code writing will address others … but I will also claim, even though people seem to deny it, that doing some good regression testing early on will save a lot of time and frustration through the whole project.
2. Robot dreaming. If you can get people to accept that testing is a good idea then:
   • They will start to dream about a wonderful, magical world where fairies come and do the testing for them. But not many IT people believe in fairies anymore.
   • Having accepted that magical test fairies do not exist, people will instead dream of robots doing all their testing for them. So they fire up Google and start searching for wonderful sounding robots (often called automated testing tools) to use.
   • But it takes a lot of time to Google for the robots and it takes longer to install and test them even if you do start to use one.
   • So someone still needs to set up the regression testing for the robot and someone still needs to do some manual regression testing until the robot is ready. This causes some sorry and a brief attempt to go back to step one “just for now”.
3. Tester and intern persecution.Finally we get to the real problem. We all know we should do regression testing but nobody in the team wants to be the one to do it. So now we finally start to tackle the problem.
   • The team instantly starts a search for interns.  If they find them this is fantastic because we can now inflict all the horrible regression testing on them.
   • If there are no interns then the team search for graduates, “business people who want to become BA’s” and other victims. 
   • If there is nobody inexperienced enough to trick into doing the work, then finally the team dump the whole thing on the usual victims in IT – the testers. After all regression testing is a testers role isn’t it?
4. Pointless suffering. Having been forced to do regression testing, the victim now sets out to prove that regression testing is a horrible waste of time.
   • The victim starts doing some simple testing by running the same test on the same functionality multiple times.  This finds few if any bugs.
   • The victim then starts running some functional tests on new functionality until reminded that this is not regression testing.
   • The victim finally, grudgingly finds some simple test cases and starts to run them each week for an hour or two.
   • Running the same simple tests for a couple of hours results in very few defects since what is being tested is usually the unimportant stuff or the stable stuff that nobody is breaking.
   • So the victim concludes that they are spending a lot of time for very little return and then finds other more important work to do.
   • The team return to step 1.

But rather than the “cycle of pain approach” I think it might be better to use scenario testing as a way to do regression testing.

The main reason is that this is the first approach anyone taught me and it worked. So I never really learned any other approaches. Not the best reason I guess, but a good enough one for me.

But the process I learned process involves more than just writing some test cases and running them. So it seems like more work, but in fact it is not. I hope to show that the approach I learned is actually quicker, more effective and more fun than either the “cycle of pain”.

The whole approach consists of four steps:

1. Understanding the context. Regression testing involves testing whether you have broken the functionality that was already there. So it stands to reason that you will test better if you understand what is already there, if and why it matters and who uses or cares about it.
   • Don’t panic.  This can be done easily. You can read the project mandate (or project charter) and you can talk to people.
   • In addition you can use some simple techniques to improve your understanding. I would suggest these as a pretty good starter pack
     • PAC
     • Using the international standard for being scared (FMEA- asking what could go wrong)
     • Creating a product summary statement
2. Scripting. You can do effective regression testing using only exploratory testing. But if you are doing the same tests repeatedly then it is probably worth writing down what you are going to do to save time when you are doing it.
   • Don’t panic. Some people think you need to build an entire library of scripts before you start testing. But you can start with a few basic ideas and then build on them each week as you go.  I will publish a guide to creating both slack and good scenario tests soon.
3. Making a map. You can run a lot of manual tests, but if all the tests are covering the same thing then you will be wasting a lot of time re-testing what you have already tested. You will also be consistently not testing the same things every week, which seems a little dumb.
   • Don’t panic. Your first go at this can be a simple list of the 3 or 4 scenarios you are testing to start with.  But as time goes on you will find it more and more beneficial to have some kind of map linking your scenarios to the things you should be testing.
4. Running some tests. Of course the point of the other 3 steps is to help you run some tests, so this step is pretty obvious.
   • Don’t panic.  You can run some tests this week and some next week.  As you go you will update your scenarios and your map so that you get more and more effective at testing in less and less time.
   • As you are running your tests though, you will learn more about the system and you will find that the context in which you are testing is changing. So go back to step one.

I guess the choice is up to you and the team.  The traditional approach (the cycle of pain) has been used for years by many experienced teams, so it is tried and tested. On the other hand, the creative testing cycle I have described here sounds a little radical and may not be as widely used. But I have found the creative cycle to be less painful, more interesting and more effective for me.

The problem with production support is that nobody has time to ask for what they want, but it is all urgent and super critical. So the last thing you often feel like doing is to slow down and understand the context that the user is in (annoyed, relaxed, sitting in a cafe, in their most important sales meeting ever etc). Instead the focus is usually on fixing “it” before you know what “it” really is.

But this often leads to rework. So whenever I am doing enhancements I always spend a little time understanding why the enhancement is needed, who it is needed by and when/how it is likely to be used in the real world. To do this though, I think we always need to spend a little time understanding the people who will use the system and why they need something new.

There are many approaches to doing this, but one I often find useful is “PAC” or “People, Activities and Context”.

PAC refers to a series of steps to understand:

• The People using the system;
• The activities they will perform; and
• The context in which they will perform them.

The steps are quite straightforward:

P: People (Define the users)

Let’s start by breaking “the users” into three groups:

• Those who actually use the system for the purpose we built it for are the Primary users.  Most regression testing is likely to focus on these users;
• The often forgotten Supporting users are those who interact with or use the system for the purpose of helping the Primary users to use it. For example production support or on-site support engineers; and
• Other stakeholders are those who do not directly use the system but who gain a benefit from the system or have a say in what it does.

Example primary users might include:

• Experienced and inexperienced users
• Wholesale and retail customers
• Users of different versions of the software – Basic Edition, Advanced Edition and Enterprise Addition for example.
• Local users and overseas users
• Onsite users, home users and mobile users

In each case you list the types of users and then ask yourself:

• Do they use different functionality?
• Do they use the same functionality differently or need a different experience using it?

A: Activities

The next step is to list the typical activities that the user might perform, that might impact or involve the system.

The trick here is NOT to just go through the requirements. Instead the focus is on the user and “what they will do during the day”.

So we might have a requirement for a user to login and another requirement to process member enquiries. But we might actually put down the activities of “talk to customers on the phone” and “send a report to the boss”.

Sending a report to the boss might involve running a report from the system (and thus match a requirement) or it might involve collecting information from several sources, including the system we are building.  So the user might actually be logging into our system to run some general queries or to just look at some screens.

Similarly, several users might do the same activity, or one activity might involve multiple users. So don’t worry about linking the activities to a specific user yet, just put them all down in a list.

So the idea is to capture “what activities does a user get involved in”.  These can then become the basis for requirements, test scenarios, stories, use cases or whatever you want.

C: Context

The final component of PAC is context.  This is important because the same user might perform the same activity very differently at different locations, using different toolkits or even in different emotional states.

So context can cover anything about the user’s environment or situation, for example:

Physical environment:

• Will this be done at the office, at home or on the road.
• Some users might be in a major company while others are in a small office.
• Some users might be in the outback in Western Australia while others are in a major city in Japan.

Technical environment:

• Will all users be using the same operating system, browser version etc. I find in particular that customers assume we support all browsers in all versions but IT. departments only test with one browser, assuming any others are unsupported.
• What hardware will people be using?  Will this impact their response times or ability to see our graphics.

Emotional state:

• During a disaster might be a different process to relaxing at the office.
• The user might be using the system in front of a complaining client or after work.
• when there is time to wait for reports to run.

Time

• Is there a specific time when things are harder or more urgent, for example peak hour or the annual enrolment week at university where things will be both more important and more likely to fail;
• Are there blackout times like overnight backups, annual reporting periods or change freezes?

Bringing it together

Now that you have collated all the information you join the dots. So you look at each user and ask yourself whether they will do each activity and in what context they will be operating.

You can put this all in a table if desired:

Of course completing the table is not the point.  The real point is to use this information to create stories, requirements or tests.

Unfortunatley there there is often little or no automation in place at all when you join a new team and there is often not enough time to get it working properly before the next release. 

In fact though, even when there is a lot of automated testing in place, I have sometimes been surprised how quickly a good business person will find a significant bug by simply sitting down and mucking around (playing) with a system for an hour.

Exploratory testing

So I think there is a place for “mucking around” testing every week on a project. This informal approach will test for the things the team never thought of and will often reveal surprises even if it is just a developer or random business person doing the testing.  And often, those surprises will be missed by consistent and formal testing even when it is properly automated.

Of course when a real tester does “mucking around” testing they like to call it “exploratory” testing to make it sound more legetimite and scientific.

Exploratory testing is, in fact, exactly the same thing as “mucking around with the system” to see if it breaks. But the exploratory testing done by a master tester, compared to a random business person, is like the difference between the work done by a master painter and that done by an enthusiastic child.

I will not attempt to cover the numerous techniques, heuristics and instincts that a master tester (or painter) needs to learn to master their craft, but I will say that regardless of whether you are an expert tester or a rank amatuer, exploratory testing can be fun and surprisingly effective (yes – even for IT Managers like me).

I will also say is that any effective approach to testing, including regression testing, involves an allocation of time to “just mucking around”. This often opens up problems you didn’t want to find, but that is better than going live with the same problems.

A good description of exploratory testing can be found in the following article if you want a better definition http://www.satisfice.com/articles/what_is_et.shtml.

Exploratory testing can be boring if you just press the same button all the time. But it can be more fun if you just muck around with the system and try to do “the stupidest possible thing a user can do”, or “see what would happen if you tried to enter Rupert Murdoch into system and see what the process will do with a salary of $120m or so per year”.

If you allow yourself time to be inventive and creative, then you will probably stumble on a surprising number of bugs … and potentially even requests for scope changes.  So this can be a pretty effective approach and failing anything else you will find some of the bugs AND you will find it will even be enjoyable if you do it properly.

But even then I think we can do better.  So there must be another option, somewhere between using robots to do your testing (ie automating testing) and mucking around having fun breaking the system (exploratory testing).

So my next quest is to find something beyond exploratory testing that will still add value when manually testing and will still not suck too much for those doing it.

Bad developers see problems and sit there waiting for someone to come up with a solution in enough detail for the developer to transcribe the solution into code, much like an old fashioned typist takes dictation and types it onto a page.
So if a bad developer noticed that their house needed cleaning, then he or she would simply complain that someone should clean it. Then if you point out that it is their house that needs cleaning then they will either claim management won’t let them clean or that the problem is more complex than it seems cannot be solved.

In fact even if you ask them to try and clean, they will just start to reveal that cleaning is “more than vacuuming” and could involve the removal of micro-particles that only quantum physicists could possible manipulate. Indeed, they will contend, it is unlikely that anyone really cleans their house and the only practical solution would be to upgrade to a new cleaner house.

But good developers are different. A good developer will notice that the house needs cleaning, work out that actually cleaning it less fun than designing a better way to clean houses and immediately begin working on the design for a new robot.

In fact, rather than cleaning the house for an hour, most good developers would prefer to spend a whole day building a robot that could clean the house for them.
Of course, the robot might occasionally turn evil and attack the developer as happens in every science fiction movie; but even this is seen as being preferable to cleaning the house manually.

So a similar thing happens when it comes to testing. Bad developers seem to think that testing is optional, or at best a good idea in a utopian world, but simply impossible with our existing systems.

Good developers suggest that we hire a cleaner to come in once a week (er – a tester I mean). But that means that our house gets dirtier during the week and is only ever clean for a short time. It also turns out to cost money to hire both testers and cleaners and it turns out that if they are not good at their job then they don’t help us to clean things up at all.

So it is no surprise that when I asked my developers to do some regression testing recently, they immediately recommended hiring a regression testing specialist. When my budget ran too low, they immediately started to build robots to do the testing for them.

But the robots turned out to need maintenance and development and sadly even testing – they were developing their own bugs. So now we face the challenge – should we build a new robot to test our existing testing robot?

Now it seems we face an infinite loop, with ever increasing armies of robots being built to test and protect us from the robots we have already built.

Or, maybe we can do some manual regression testing.

Manual regression testing involves somebody sitting down and testing that what we built yesterday still works, even though we added some new stuff today.
Because regression testing is boring, we would like to defer it to the end of the project, but because our system gets dirty one day at a time it seems better to test it every day.

But without robots and nanotechnology we cannot test everything every day and even with robots, they only test the same obvious stuff every day.
This is what you would call a problem that sucks:

So I decided to as a tester – What is it about testers that makes them enjoy doing long hours or boring, sucky work?

The answer might surprise you – the tester I spoke to actually hates doing long hours of sucky boring work. He even hired a cleaner to clean his house because he could not afford a robot with nontechnology. Also he noted, he would not trust a robot that was built by developers because he was sure it would eventually turn evil and try to exterminate humanity.

So I asked him a new question – how can manual regression testing be anything but sucky work?

“Build some robots,” he responded, “Automated testing is much better than manual testing for fun value, even though manual testing often reveals more important problems.”
“But how can a human out-test a robot?” I asked, “ and why do you do manual testing if it sucks.”
“Repetitive testing sucks and robots are better at stuff that sucks. But manual testing is really problem solving not just typing test scripts,” He responded.

My immediate thought was that if I could fool the developers into thinking that testing was fun then I might be onto something … but they are probably too smart for that.

So the bigger challenge is to try to find ways that testing is actually about problem solving while keeping the repetitive stuff to a minimum. Unfortunately that sounds like the opposite of regression testing doesn’t it?

So – how can regression testing be valuable work that people don’t hate doing – rather than low value work we defer indefinitely? (Besides through building robots).

Interestingly it turns out the real world is harder and more ambiguous than the projects in my training slides

Having said that though, the fundamentals don’t seem to change. We have stopped a project and are about to restart. It is really urgent and we have inherited a project whose budget (in time and money) has already been spent. So we really need to get going.

But are we better off starting or are we better off getting our act together before we start so we are not “mistaking activity for progress” by rushing off in the wrong direction?

What I say in my training courses is to get the team together and brainstorm everything that needs to be in place before we start.  Then prioritise these ideas using MOSCOW and spend your first iteration (or sprint) nailing those stories.  Then if you have not done all the “must haves” then don’t start and if you have not done all the should haves then you know your next retrospective will include “should not have started before …”.  But with your should haves you can turn them into risks or tasks to do in the next iteration or two by dropping some stories to make room for them.

Great, I can’t fault the logic, but now I am in the real world and the development team is already off and running on some of the things they know need doing for our first urgent release, while some of the team are yet to arrive on site.

So here are the must-have’s I am applying.  Let’s see what happens.

1. Do we know where we are going?
• Do we have a Mandate to do the project? Ie
• Do we know the problem we are solving (yes). 
• Do we have funding allocated and not just “soon to be sorted”. (Yes)
• Do we have a complete backlog that the whole team understands (no).  Eek … do we know what to do for the first release? (Yes – we need to be ready to integrate some functionality for the other key project by the end of next month.
• Do we know our scope and priorities? (Yes-ish, er that is to say roughly yes) [which is probably the most dangerous answer of all].  (But we do know the scope for t he release in one month and we know our first priority is to hit that deadline subject to our other first priority of sufficient quality is mandatory).
• Have we defined success? Do we know what “good” and “fixed quality” actually mean? When is a story done? What non-functional requirements need to be meet to release? (Er No we are not really clear yet)
2. Do we know how to get there?
• Do we know what practices we are using? (Yes – we are stealing them from the project we need to integrate into).
• Really? Do we have planning and showcase meetings booked (yes). Do we have retrospectives and standups? (Yes)
• Do we know what technologies we are using? (Yes, except for some of the bits).
• Do we know who is doing what on our project? (We have a sponsor, a product owner, a scrum master, a BA, some developers and a project manager). Again – Do we know who is doing what? (They should do, they know their job titles, except the Scrum Master, I am not sure if we told him that he is on the project to the bitter [or joyful] end). In other words – We need to be clear on what they actually do. Cool – do we have an architect and a tester? (Long pause … oops)
• Do we have a test strategy that is understood by the whole team? (No).
• Do we have a publicly available risk register? (No but the PM is listing and discussing the key risks)
• Do we have a high level architecture that is understood by the whole team (coding standards if we want them, standard tools, guiding principles and of course a big picture on a wall or whiteboard of the context diagram and dataflow diagram (or domain model or strawman or something). (Yes-ish)
• Do we know the biggest constraint that is likely to inhibit our success? (Yes and no)
• Do we know the few things that really need to be done well and the other things that need to be done well enough to succeed? (sort of) What has to work for us to succeed and what has to be done really well? What will be forgiven and what will be remembered after the project is done? (er … yes)
• Do we have a road map of what is being released in what order, say epics per release? We have lots of stories and epics in Sharepoint, Jira and Version One. But we do know what needs to be done for the end of the month and we are consolidating to one backlog in the coming days)
3. Are we set up to succeed?
• Do we have the right people? (yes)
• Have we moved to one location (No – same planet but different countries)
• Have we cleared the decks of all other distractions (no – some of the core team are on other initiatives)
• Do we have real buy-in and trust from both the team and the business stakeholders (yes)
• Have we all worked together before (no)

The trainer in me is screaming stop. The enthusiastic optimist in me is calmly saying we have been here before and the crew is a good one so we will pull it off.

But the project manager in me is comparing risks against each other to pick the least worst (or should I say best) options. (I am not the PM on this project but I will sink or swim with him so my inner PM is talking loudly in my head).

So here is my current cunning plan:

• Get the test strategy in place before anything else.  Then make sure we have a technical scope and high level architecture diagram that we can show both the related project and the business customer.
• Define Done, Quality and the “what do you really do” part of the key roles
• Break the project into “imminent release” and “rest of project” and get on with some of the imminent release while sorting the rest out.
• Smile and look confident while copping a schalaking from the sponsor for not being fully ready.  Calmly repeat “its better to be unhappy now than at the end of the month; It’s better for the team to commit than to force them forward before they are ready; We know the biggest constraints and we are focussed on managing them first; stop hitting me, its not really appropriate).
• Expect to only deliver one quarter of my potential velocity in sprint one because I am really doing sprint two late.  Maybe I should delay to sort things out but that always seems to rub me the wrong way.What

What would you tell me to do?

• Who I am communicating with?
• What should I be telling them?
• How should I communicate with them?

Even thinking about those three questions on the bus on your way to work might help create better communication. But I thought I would break the questions down to come up with a slightly more complex plan that is still not hard to do.

Who am I communicating with?

The first step in building a communication plan is to perform a stakeholder analysis. But this does not need to be complicated.

Impact on us and impact on them

One approach is to identify the stakeholders as a team. I like to enhance this slightly by comparing stakeholders according to:

• How much impact they might have on us; and
• How much impact our project is likely to have on them.

This helps the team understand the different levels and types of communication that might suit different stakeholders:

• Some people have the power to veto or delay our project, or control the resources we need access to. So we need their help and we need to explain why we they should support us, especially if they do not benefit from the project.
• Some people will not be impacted by our project and we don’t really need their support. So we can let them know what we are doing but we shouldn’t really harass them too much with information noise.
• Our project might have a significant impact on some people who can’t do much to stop us. So we can steamroll our changes through without their support. Unfortunately the concept of project Karma suggests that this will bring us bad luck in the future. So we should probably explain the project to them and seek their help in defining and understanding the implementation and support needs that our change will require.
• Finally, some people have a huge impact on our ability to deliver the project and will also be impacted heavily by the project. These people want to know what is going on in more detail and also have a lot to tell us. So we should probably keep them close to the project and communicate with them very heavily.

What the stakeholders are interested in

Once I have listed my stakeholders, I still need to know what to say to them. But I don’t want to send the same email to every stakeholder, so I classify my stakeholders by the type of communication I want to provide.

One simple approach to doing this is to create the following table and answer each of the questions (at least for key stakeholders). I often do this as a team but sometimes I sit down and do it on my own.

For completeness, I often add a column for “impact on us” and one for “impact on them”.

How and what do we communicate?

I will leave the specific details of what you communicate to you, because it will depend on the project. But here is how I often compile my own communication plans:

This plan is obviously based on my stakeholder analysis and on an understanding of what I want to communicate.  The level of detail will vary but I find it very useful to share with the team who we are communicating with, when we are doing so and who in the team is responsible for doing it.

It a simple approach but has proven sufficient for many of my projects. Let me know if it works for you.