Regression testing day 2

This article probably makes more sense if you have read regression testing day 1. The aim is to give you a possible way of building good regression testing on a project one day at a time, while testing as you go.

First thing in the morning – Refresh the context for 20 minutes

You probably finished yesterday with a review of what you learned on day 1. If you didn’t, then do that review now.

Now, before you start, you want to come up with a brief plan for the day:

• Discuss which scenarios you want to run.
  • Are there some scenarios waiting for bugs to be fixed?
  • Are there some scenarios that you can rerun with different data?
  • Are there some that you didn’t get to yesterday?
  • What new scenarios could you add today?
• Discuss and questions you have unanswered from yesterday.
• Decide on a mix of exploratory and scenario testing for today. For example you might choose to spend 40% of the time you have available on exploratory testing and 60% on scenario testing.

Write the scenarios and questions out on post-it notes or index cards.

Step two – do some scripting

Consolidate the scripts you have already done. You might find that you can combine two or more together to make a single scenario. Similarly, you might find that several of the scripts are re-testing the same things.  They might all involve logging in for example, or several of them might involve accessing the same customer details.

See if you can remove 25% of your scripts through consolidating them.

Now, come up with a couple of additional scripts that

Next, think about whether you would like to come up with some standard data to reuse. You might create some of that data now, but you can also create it as you run some of the scenarios.

For example:

• If one of the scenarios involves creating a customer, then you might record the data as you go and use the same customer when you run other scenarios; or
• If you are going to be running the same scenario a number of times, then you might run some reports (or customer statements) when you run it the first time so you don;t have to recreate the data next time.

Step 3 – improve your “maps”

Yesterday you created some rough scenarios and used them to do some testing. Hopefully you also set up the basic management report which showed the scenarios you have run and whether they passed or failed.

So the first thing to do is to reset the management report to show that no scenarios have yet been run today. But the table I showed only shows if something has been run, not how often it has been run or how long it has been since we last ran it. So it might be good to split out table in two.

• Our scenario map from yesterday will evolve into a reference tool containing scenarios, data and things to test.
• The daily map will show what is going on today.

New scenario map

From now on the scenario map will be a list of scenarios, along with the information you want to track to have access to in order to make your testing easier.

This will evolve over time, but for now we can make do with a modified table:

Scenario	Last run	Test user	Other data	Failed
1	1 March	Bill
2	1 March	Mary
3	2 March	Mary

 

From now on, you might not run each scenario each day. So it might be worth recording the date you last ran the scenario.

You might also find it worthwhile listing the user name for each scenario to keep track of the different users that you log in as each time you run a scenario.

You can retain the passed and failed columns or you might record them on the “test story wall” described below.

Assuming you list the test users then you might also create a card (or record) for each one, including:

1. Test user name
2. Role type and (if applicable) persona
3. Login details – name, password
4. Browser type
5. Software version

Other data might include customer details or other information you will enter often enough to make it worth recording.

New daily map

The daily map is really just a story wall for tracking the tests you are running and the questions you want answered. It is useful if you are testing on your own, but it really comes into its own if you are testing with a group.

Create the following story wall to track the tests as you run them:

Coming soon	In progress	Failed	Waiting for retest	Passed

Write each scenario you plan to run on a separate index card and place them in the coming soon column.

When you start on a scenario, place it in the “in progress” column. Or if you want to cut corners, take the card with you while you test.

• If the scenario passes first time then place the card in the “”passed” column.
• If the scenario fails, place a 1 on it and put it in the failed column.  Now record the bugs you found so they can be fixed.
• Once the bugs you found in a scenario test have been fixed, move the card to the “waiting for retest” column.
  • If it fails again, put a second 1 on it (to show it failed more than once) and place it back in the failed column.
  • If it passes then move it to done.
• At the end of the day, record the total number of “fails” and the total number of passes and write the totals next to the board,

But in regression testing, you will come up with questions as often as you find bugs. So I think it is worth also having a register of questions and decisions. To do this, read the article on “a risk register for lazy project teams” and use the same format to create a register of questions and decisions.

The format will look like this:

Open questions:: (place index cards here with questions or issues)

Closed questions: (Place index cards here when they are resolved)

 

Assumption/decision	Validated
Assumption one
Assumption two
Assumption three

 

Go and do some testing

Finally, of course, you should probably do some testing.

So go and run the scenarios you are planning to run, just like yesterday. And just like yesterday, finish the day with another debrief.

The only differences should be that you have slightly better scenarios and slightly better tracking.