Actually I made it up so it is not too well understood outside of my own loungeroom.  The approach is essentially the same as the 7-S style of assessing the interaction of the multiple systems, skills, stucture and other elements of the team’s whole ecosystem. But this is a little different because we start by looking at when (and why) the team’s internal or external customers interact with it.  Then we assess the team’s ability to support those interactions.

Moment of truth(n)  a moment when a person or thing is put to the test

Collins English Dictionary as quoted at www.thefreedictionary.com

Any interaction with a client is “a moment of truth” for the team. It tests the connection between the team’s value proposition, strategy, implementation, staff, skills and systems and it generates the experience that lasts in the customer’s memory until he or she interacts with the team again.

A “moments of truth analysis” therefore starts by identifying the interactions a team has with its customers, stakeholders and potentially vendors.

When do customers interact with the team?

Starting this analysis is quite simple, although it could potentially be time consuming.

1. Find out what products or services the team provides to customers or stakeholders. This would involve their official products and also complaints, requests for information and other “back-office” approaches.
2. For each product and service, identify the times and context in which the customer interacts with the team. These are the “moments of truth”.
3. For each moment of truth, find out why the customer is interacting with the team and what they want to achieve out of the interaction.

What happens when customers interact with the team?

The next step is to understand how the team interacts with the customer in the moment of truth.

• Who initiates the contact?
• Is it a one off thing or part of an ongoing process?
• What happens?

Once you know when and how customers interact with the team you can map these into the series of interactions that typically create a “customer journey”.  This is the simply a typical accumulation of interactions the customer has with the team.  For  an internal accounting support team the journey of one of their customers (department heads) might be:

• Establish budget
• Receive monthly report
• Query strange amounts
• Request ad hoc report
• Prepare next year’s budget

Mapping the interactions from the team’s perspective

This is the point where you start to analyse the things that may be impacted by any changes you make to the team, as well as anything that could go wrong or be improved.

For each interaction you can ask questions such as:

• How often does this occur?
• How long does it typically take?
• Which other interactions will potentially impact this experience?
• What is the context in which this interaction is taking place?
• What was the last interaction the team had with the customer and typically how long ago  would that interaction have been?
• What inputs does the team need to support this interaction?
• What comes out of the interaction – from the team’s point of view and the customer’s?
• What skills and knowledge does the team member need to deliver this service?
• Who in the team is involved?
• What guides, manuals or tools does the team member rely on?
• What measures or service level agreements does the team have in place?
• What might go wrong?
  • What would cause this?
  • What would happen if it did?
  • What would happen next?
  • How might any potential changes make this interaction better or worse? Which of the above aspects of the interaction could potentially be impacted?  Which could negatively impact the potential change?

Assess the supporting processes if relevant

Once you have mapped out the moments of truth you can also map the internal processes that the team performs in order to be ready for those moments of truth.

Wrapping up the moment of truth analysis

The final stage is to consider how each of these elements interacts with the others to support or hinder the team’s ability to provide positive “moments of truth”. This can be combined with the 7-S or arenas of change approaches to understand how the team is currently set up to provide its services and how resilient they will be if and when you implement a particular change.

In fact SCRUM and agile approaches even try to define the bare minimum number of meetings that are needed and only have them (Actually one of my friends claims that they have removed all the meetings – “In agile we don’t have meetings, we only have workshops.  Meetings are discussions and Workshops produce something tangible each time”).

Yet the reality seems to remain that meetings (or workshops, or gatherings, or war councils) often end up getting in the way of doing the real work:

So what would you do if you had to look at the return on investment of each meeting?  Would they actually stack up from the point of view of making money for shareholders, making life easier for the crew or improving the experience for our customers?

I think it would be fairly easy to classify most of the meetings we have as:

1. Adding value – this meeting makes life better;
2. Necessary waste – I wish we didn’t need it, but we need to do it because we don’t have a better alternative; and
3. Complete waste – I would rather take the free donoughts and eat them at my desk.

I also think this is easy to do if you look at which meetings add value or create waste if we use the following structure that I often share with business analysts. Each meeting should have three things in it:

1. The framing of the meeting
   • State the goal or purpose of the meeting
   • Clearly state what is expected at the close of the meeting (eg minutes, an agreement, an action list etc)
2. The opening of discussion
   • Creating a list of things to be discussed, each of which is a mini-meeting. In other words we should not just wander randomly through topics as they come to mind (unless that really is the purpose of the meeting).
   • For each topic or agenda item, we can therefore create the following for it
     • Frame or introduce the goal of the topic and what we expect to get from the conversation
     • Discuss it
     • Close the topic by confirming what we found out, agreed or need to do next.
3. The closing of discussion
   • This could involve agreeing Who does What by When; or
   • This could include confirming what has been agreed and how it will be communicated (ie What will be communicated When by Who); or
   • This could include a summary of our analysis – eg documenting the requirements by Who and When.

We all have too many meetings, but if think of meetings as work then we should put them down gently or define how we will get value from them. I am sure your team is already super-double-extra agile, but with my team I sometimes think we need to go back to basics.

So we can’t do better then I am going to ask my crew from now on to at least agree to only turn up to review their meetings and either

1. FOC them (define the Frame – Open – Close for them);
2. Define them as necessary waste and come up with a plan to either make them really add value or make them unnecessary; or
3. Define them as waste and eliminate them unless there is good catering (actually – eliminate them anyway and move the catering to a useful meeting).

So far we have been looking at how to do some testing, and then do some basic scripting and mapping as we do more testing. In doing so we have (hopefully) been learning more about the system we are building but our main focus is on making sure it is still performing the way we want to.

Now we are going to evolve our testing onto a proper OODA loop.  I have created another article to explain what an OODA loop is, but for our purposes it means this:

How fast and how well can the team move between the following 4 tasks?

1. Observe the world around them – the way users are operating, the way the system is performing and the way things are coming together.
2. Orient themselves or make sense of all that data (interpreting, analysing, integrating and assessing).
3. Decide what to do next with all that information.
4. Act on their decisions as a cohesive group.

To do this we need a little lesson in OODA regression testing theory

Regression testing is a series of loops not one straight line of work

With regression testing, most people think it is like a line rather than a loop:

1. Understand the system
2. Create some scripts and then make a list of them (map them)
3. Run the tests to break the system
4. Fix the bugs

But this misses two key ingredients in our OODA loop – both of which are related observing what happens when we act and then learning from it:

• By running one set of tests we should be able to learn more about the system so we can test it better next time; and
• By running one set of tests we should be able to learn more about the system so we can use that knowledge to update our requirements, create training material and pass what we have learned on  to others.

So the real process looks a little messier when you first look at it:

But that looks like a lot of work

Fortunately it is really what we have been doing in day 1 and day 2. So for the next couple of days we will simply repeat what we did in day 2, and add these questions:

• Should we improve the way we capture and track bugs?
• How do we pass potential new requirements? Who should decide whether to act on these?
• How can we improve our test scripts and maps?
  • Should they be more sophisticated?
  • Should we drop some in favour of more exploratory testing?
• How do we pass on what we are learning?

Each of these questions relates to one of the Act boxes in my diagram and you will notice that each of them then becomes another OODA loop within the model.

My suggestions for early improvement

I think it is a good idea to start writing out the most common or critical test scenarios in a “Use Case Like” format.  This turns out to be a good way to pass on information for training, a good way to automate and a good way to show people what to do when they are testing the ssystem

I also think it is a good idea to at list start collecting a list of questions and known (or expected) answers.  This is often done in the format of an FAQ.

I think it is a good idea to write down the work arounds your are finding. If these never get fixed then you have some warnings to include in the release notes. Of course, they might be fixed later but even so it will be useful to remember them as you continue testing.

Finally, I think you will realise that  your are finding several “trivial bugs” that the team don’t want to fix. Rather than tracking these as defects or “unlikely to do requriements” you could also record them in an FAQ like format so that they become known errors or, in common IT terms “Unexpected features”.

So you can see that I don’t see regression testing as breaking the system. Instead it is the engine that is driving our continual learning (and hopefully improvement) of the system.

In those days requirements were really easy for me: people would come to my desk and ask for something, or they would email me or maybe even leave a scribbled note on my desk. There were no standard formats, no formality and (usually) no problems.

But even back then I had to work with vendors and sometimes that was when the trouble started. Some were happy with my “give me a call” approach to requirements, while others required a ticket and some even required a complicated work request form.

Now days, some production support teams are more professional that I was and they actually have real requirements or (if they are agile) stories. And most of them seem to be flooded with vendors.

So, do stories work with vendors?  The obvious answer is YES. For example, if you receive a story in the following format then it should work for you and also for your vendor:

• As a coffee club member I want to be able to see a list of future coffee appreciation classes so that I can enrol in those that look interesting.

But what about complaints and bugs?

• As a coffee club member I don’t want to have my name spelled incorrectly because it is really annoying me.

I guess that works, but it is not how the user will raise the request.  They are more likely to say “you got my name wrong”.  We would then ask “what should it be” and then just capture the requirement to “for member 1234, please change the surname from Jonnes to Jones”.

That would work OK. But when there are multiple vendors all connecting different pieces of the puzzle then things can unravel surprisingly quickly.

For example, if you want a list of upcoming coffee appreciation classes, but you have outsourced the CMS to Kingcorp international (not a real company) and you have got need to link the classes to your existing database, built by the internal team, then you might end up with:

•  As a coffee club member I want ….
• As the CMS, I want the cafelate database to provide me with data in an xml feed so I can send the information needed to the screen so the coffee club member can ….
• And so on.

But rather than butchering your stories so that you can use them, why not only use them where they are useful.  Here is my starter kit for stories and other formats that work well with multiple vendors and teams.

Each tool is designed to either clarify the value you are trying to create for your customers, or remove the ambiguity involved in what you are asking for or both.

For reference, this is how you use each tool:

Problem statements and focussing questions are contained in this article. They are essentially an overall statement of what you are trying to achieve.

Use cases and stories

Use cases are roughly described in this article and you can also google the term to find a lot of information on them.

User stories are all the rage in agile circles. They are essentially a summary what you want, who actually wants it and why. So the standard wording is this

• As (a type of user) I want (something) So that (the reason or benefit I get).

Functional requirements

These are statements of what you want a particular system to do. They are described in this article and might follow this format

• (when something happens) (the system )shall (do something) (if this condition is true)

They work best when complimented by non-functional requirements (the qualities you want like user friendly or reliable). So you might use the FAB format I described in a previous article.

Acceptance tests

Acceptance tests actually list the conditions you will test for.  The most common format at the moment seems to be “given- when-then”.

• Given (an existing thing is true);
• When (something happens);
• Then (this is what should happen).

Talking

There are several international standards for talking.  I usually use English with Good Manners, but there are many other standards available.

Talking to your vendor still seems to be the best approach to me, even if complimented and clarified by the other tools above.

The only short-coming that talking has is that it does not work well when “nerds” and “business people” have different tact filters (see here http://www.mit.edu/~jcb/tact.html). This occurs when you are talking to your account manager but you are not letting your techo’s talk to their techos directly. Problems get solved and solutions get improved whenever you let the people in each team talk directly.

An example:

So what?

So that brings me to three quesitons:

Is there a best way to communicate your requirements with vendors on agile projects? Yes – stories and lots of conversations. Unless you are in different timezones then more elaboration around the testing will help – detailed use cases, acceptance tests or even old school requirements might help but rely mostly on the story and  conversation.  Unless you know more about your own team than I do, in which case common sense trumps advice from blogs.

So the same answer applies as per projects? Er – no.  I actually think Prod support teams have a lot more one line requests to fix certain things out of t he blue. This would be fine if people asked for what they needed and production support teams understood what people asked for. But they don’t, so they should talk to their customers.

But when you deal with a vendor and requests come out of the blue it is worth reminding them of why you want something AND removing ambiguity.  Conversations still win over anything else, but it is often worth using a more structured approach using tools like the ones I explained above in addition to the stories and conversations you are using.

I hope that is useful. Let me know if it works for your team.

But what is a use case? It is simply an example of how a system (or business service) could be used by someone.

Where do Use Cases come from?

Use cases were invented by a Swedish guy named Ivar Jacobson, who is really famous in geek circles. I imagine he has his own groupies and he certainly has a big fan base, because he apparently invented sequence diagrams, he was one of the founding creators of UML and, if people had listened to him, he would have made the whole IT industry pretty simple and straightforward. If you google him you will find out that he is still trying to get the IT industry to admit how simple things could be, while the rest of us are still trying make it all sound complicated so we can continue getting lots of cash for doing obvious things.

According to James King history (I haven’t checked my facts too carefully) people were building systems by asking the users what the system should do. But when they built the systems they didn’t work too well because nobody had thought about how a user would actually use them.

So Ivar said to his friends “why don’t we actually ask users how they would use a system”.

But his friends were confused, “how would we do that?”, they asked.

“Let’s just ask them for some examples of how they would use it and then we can get the IT geeks to work out how to build something that does that for them”, explained Ivar, “We will call it ‘Using Examples to Build Stuff’”’.

Unfortunately for people not living in Sweden, Ivar speaks Swedish and apparently they have a word that means “a case (example) of how something would be applied or used”.  The unfortunate bit is that when Ivar’s simple idea was translated into English, the translator decided that Use Case was the closest thing we have in our language to “a case of how something would be applied or used”. So instead of having “examples” or “ways you could use this” we have are stuck with the slightly less obvious “use case”.

Of course if you just ask for a couple of examples of how people might use stuff, then you will have a lot of holes in your requirements. But Ivar had thought of this.

Use cases for really simple systems

For simple things, he suggested listing the users (who he called actors) and listing the examples of stuff they might use the system for (the use cases). He then invented two ways of communicating this.

An obvious approach (called an Actor-goal list) is to simply create a table listing the actors and what they want to achieve:

For a simple system, this might be all the documentation you do to create a list of test scenarios. But Ivar and his groupies also came up with a diagram to show the same thing. They drew a box representing the system they were building or testing put all the use cases inside the box. Then they drew some stick figures to show the users (actors) and then drew lines to connect the actors to the use cases they might use.

This provides a map of the use cases, or more technically a “Use Case Diagram”:

Use cases for more complicated things

Ivar and his groupies, together with other gurus came up with some advice for how to better describe use cases when you want to know more about them. Unfortunately there are now several different international “standard approaches” for doing this, but fortunately they are all pretty similar.

So here is the internationally agreed James King version of a use case when you are using them for scenario testing.

Firstly, we will break the use case into four components

1. The metadata (information we want to know about the use case)
2. Sample data to use (this is not generally done for use cases, but it might be useful to us)
3. Instructions – The happy days scenario (a series of steps to follow when things go well)
4. More instructions – Other scenarios (examples of where things go wrong or where people use them differently)

The metadata

You can add whatever you want here, but these are my suggestions:

You could also add things like “expected time for the scenario to complete”, a picture of the screen the user should see or reference documents that the tester can read for more information. It is really up to you.

Sample data

If you run the same tests a lot then you might want to record the information you will frequently enter. For example you might record:

Of course if you are really sophisticated you will have a lot of personas (sample users and members). In that case you would simply list the personas to use,

Instructions

This is where you actually write down the steps to follow when running the use case. You could go into a lot of detail if you want others to be able to use the instructions, or you could keep it really simple if you were the only one using the scenarios for testing.

When talking to use case groupies you would say that a use case consists of several scenarios. Each scenario is a different possible outcome for the use case. For example the “happy days” scenario is the one where everything works properly and the actor makes the simplest possible choices.

So if you only document one scenario in the use case (ie one example of how the use case might go) then it should be the happy days scenario:

The step number allows you to record where things went wrong. The instructions let the tester know what to do and the expected result allows the tester to see if the right thing happened.

In a formal use case you would actually add a step for what the actor does and then a separate step for when the system responds. This makes it easier to identify specific errors or different outcomes. I don’t care about this for my test scenario but if I were working on a system that could kill people (say an autopilot for a plane) then I would definitely use a format more like the following one:

1. The user presses the “add new guy” button
2. The system displays the member creation form
3. The user enters the name and presses tab
4. The system validates that the member does not already exist and takes the user to the email field
5. Etc

Again, I should point out that the amount of detail you add will depend on who will be running the scenario. You might just say “add the member” or you might do really detailed instructions so anyone on the team can follow them.

Of course when I run this scenario I won’t always follow the instructions correctly. In fact I might turn off my machine at step 3 (or at least close the browser). Then I will log in again and try to enter the member again to see what happens.  This is where the tester can use the international standards for being scared and being stupid.

But the first time I run the scenario I will probably try to follow the instructions and make it work.

Other instructions

In addition to the happy days scenario, you might enter some alternative flows or other outcomes.

For example, I might create an alternative to test what happens if Mr Smith already exists in the system. In this case I might start deviate at step 2:

The number 2a tells me where I came from so I don’t need to write the whole thing out and then the final line should either tell me the use case is ended or that I return to step x and continue.

Complex scenarios

Use cases were designed to help build a system and not just to test it. But for regression testing you might decide to create a scenario (or test case) that includes bits from several use cases.  For example, I might create a scenario that allows me to

1. Attempt to login but get the password wrong
2. Try again and get in
3. Add two new members
4. Run the daily membership report
5. Book one member into our special coffee connoisseur course
6. Change the booking for the course
7. Confirm the member got an email on joining the club, one one joining the class and one on the class change.  Confirm the other member only got an email on joining

So that’s about it – the idea of use cases is that you create some examples of how a user might use the system.  The use case diagram could be a good map to help you understand which users are involved in which use cases and by documenting the use cases in more detail you are creating a set of instructions for other testers to use.

You might also use this approach to build automated testing or to pass information onto trainers and technical writers.

First thing in the morning – Refresh the context for 20 minutes

You probably finished yesterday with a review of what you learned on day 1. If you didn’t, then do that review now.

Now, before you start, you want to come up with a brief plan for the day:

• Discuss which scenarios you want to run.
  • Are there some scenarios waiting for bugs to be fixed?
  • Are there some scenarios that you can rerun with different data?
  • Are there some that you didn’t get to yesterday?
  • What new scenarios could you add today?
• Discuss and questions you have unanswered from yesterday.
• Decide on a mix of exploratory and scenario testing for today. For example you might choose to spend 40% of the time you have available on exploratory testing and 60% on scenario testing.

Write the scenarios and questions out on post-it notes or index cards.

Step two – do some scripting

Consolidate the scripts you have already done. You might find that you can combine two or more together to make a single scenario. Similarly, you might find that several of the scripts are re-testing the same things.  They might all involve logging in for example, or several of them might involve accessing the same customer details.

See if you can remove 25% of your scripts through consolidating them.

Now, come up with a couple of additional scripts that

Next, think about whether you would like to come up with some standard data to reuse. You might create some of that data now, but you can also create it as you run some of the scenarios.

For example:

• If one of the scenarios involves creating a customer, then you might record the data as you go and use the same customer when you run other scenarios; or
• If you are going to be running the same scenario a number of times, then you might run some reports (or customer statements) when you run it the first time so you don;t have to recreate the data next time.

Step 3 – improve your “maps”

Yesterday you created some rough scenarios and used them to do some testing. Hopefully you also set up the basic management report which showed the scenarios you have run and whether they passed or failed.

So the first thing to do is to reset the management report to show that no scenarios have yet been run today. But the table I showed only shows if something has been run, not how often it has been run or how long it has been since we last ran it. So it might be good to split out table in two.

• Our scenario map from yesterday will evolve into a reference tool containing scenarios, data and things to test.
• The daily map will show what is going on today.

New scenario map

From now on the scenario map will be a list of scenarios, along with the information you want to track to have access to in order to make your testing easier.

This will evolve over time, but for now we can make do with a modified table:

From now on, you might not run each scenario each day. So it might be worth recording the date you last ran the scenario.

You might also find it worthwhile listing the user name for each scenario to keep track of the different users that you log in as each time you run a scenario.

You can retain the passed and failed columns or you might record them on the “test story wall” described below.

Assuming you list the test users then you might also create a card (or record) for each one, including:

1. Test user name
2. Role type and (if applicable) persona
3. Login details – name, password
4. Browser type
5. Software version

Other data might include customer details or other information you will enter often enough to make it worth recording.

New daily map

The daily map is really just a story wall for tracking the tests you are running and the questions you want answered. It is useful if you are testing on your own, but it really comes into its own if you are testing with a group.

Create the following story wall to track the tests as you run them:

Write each scenario you plan to run on a separate index card and place them in the coming soon column.

When you start on a scenario, place it in the “in progress” column. Or if you want to cut corners, take the card with you while you test.

• If the scenario passes first time then place the card in the “”passed” column.
• If the scenario fails, place a 1 on it and put it in the failed column.  Now record the bugs you found so they can be fixed.
• Once the bugs you found in a scenario test have been fixed, move the card to the “waiting for retest” column.
  • If it fails again, put a second 1 on it (to show it failed more than once) and place it back in the failed column.
  • If it passes then move it to done.
• At the end of the day, record the total number of “fails” and the total number of passes and write the totals next to the board,

But in regression testing, you will come up with questions as often as you find bugs. So I think it is worth also having a register of questions and decisions. To do this, read the article on “a risk register for lazy project teams” and use the same format to create a register of questions and decisions.

The format will look like this:

Go and do some testing

Finally, of course, you should probably do some testing.

So go and run the scenarios you are planning to run, just like yesterday. And just like yesterday, finish the day with another debrief.

The only differences should be that you have slightly better scenarios and slightly better tracking.

But where do you start if you want to do effective regression testing, but you are already busy and don’t want it to be huge burden?

I hope this article and a couple that follow will turn out to be a guide to one painless way to get regression testing up and running. Let me know if it helps.

Day one – first thing in the morning

Start by getting some context before you expend a lot of energy testing what you don’t understand. I have put some tips in this article, but you can also just do the obvious things:

• Read the project charter if there is one, otherwise sit down and create one
• Sit down and discuss the users of the system you are building and the product you are building
• Document what you found, even if it is just on the whiteboard or on napkins

This can all be done in under one hour and it is worth getting the regression testing team to sit down together to do it. Of course you can feel free to take 2 hours if you want to fully understand what you are testing

Day one – 20 minutes of scripting

Create some scenarios that you will use for testing.  This does not mean that you have to fully document well written test scenarios, since that comes later.

The easiest way is to simple use the PAC table you created and come up with some examples of what some users might do.

I am using the example project from a previous blog where a cafe owner wants to :

This is our initial map for our regression testing. It won’t last long but it gives us a place to start.

Spend an hour or two doing exploratory testing

Now that you have some rough scenarios, log into a test version of the system and try to run some tests.

The tests you run will be “exploratory” because you have not created any instructions yet. But for each scenario, don’t just log in and make things work, instead try this:

• Try to run the scenario successfully from end to end and then record the outcome
  • Write any bugs or “strange things” on a piece of paper or whiteboard
  • Write down how long it took to run the scenario
• Stop and ponder for a moment, Use the international standard for being scared to come up with things that might go wrong. Then try to run the scenario again seeing if it will fail.
• As you run the scenario a second or third time, try using the international standard for being stupid to come up with mistakes or “strange things” the users might do at each step during the scenario.
• Document your findings again
  • Scenario number
  • Potential errors or questions I found
  • Rough steps I followed

Go and report the potential errors you found and if they turn out to be errors then record them in whatever bug tracking system the project uses.

Discuss the context again for 20 minutes

Sit down with the other people doing regression testing and discuss how things went

Provide the following “management report”:

Tests pass if you found no errors that went into the bug tracking system. They fail if you did record errors. Passing or failing is not good or bad, it just shows which scenarios found something.

Now sit down with the group again and discuss the context your are testing in

• What led to errors?
• What did you learn?
• Should you change the PAC or Product statement? Should you change the scenarios you are running.

There is more to do tomorrow, but you have already done more thorough regression testing than a lot of teams do. Well done.

Then I worked on a project with a mining company and one of the consultants explained human factor analysis to me in simple terms. He told me that mine sites can be dangerous and part of his job was to “stop people killing themselves when they are stupid”.

I suggested he stop hiring stupid people but he told me that they tried that and it didn’t work. Apparently you can be really intelligent on a mine site 800 days in a row but then be stupid for 10 minutes one day and be in an accident and then be killed.

“Luckily we have a standard for being stupid though” he said

I was surprised that he could get people to comply with a standard for making mistakes, but then he showed me “iso90210 (not a real standard) – the standard mistakes you can make when using a machine”.

1. Action omitted
2. Action too early
3. Action too late
4. Action too much/too often
5. Action too little/not often enough
6. Action too long
7. Action too short
8. Action in wrong direction
9. Right action on wrong object
10. Wrong action on right object
11. Wrong action on wrong object
12. Information not obtained/transmitted
13. Wrong information obtained/transmitted

It seemed like a long list of obvious things, which is what it turned out to be. I found when I was analysing processes (which I was doing at the time) that many errors could occur in the process when one of the things happened in the list. So I combined this with the “international standard for being scared” (FMEA) to find I could tear most processes apart and find errors really quickly.

Even if you can’t break the process instantly (which you usually can), you can almost always break it if you:

• Apply one standard mistake; and
• See what happens and then apply a second one rather than simply moving on.

This also applies to computer applications. For example

• The user gets to question 5 and realises they do not have the information they need to complete the form.
• The system gives them a polite error message.
  • Rather than completing the form or hitting cancel (the right thing to do) they exit by hitting the “back button” or closing the browser (right action wrong object); or
  • They hit cancel and nothing happens so they hit it again 3 times (too often) and third time they hit the button the browser decides to reload and process the click on the wrong button (something from the next screen).

I find that even though I am not a tester, I can often break applications in regression testing by assuming my users will comply with the standard for being stupid at least once or twice.

You can take things a lot further if you study “human factors” because it turns out that “human factor analysts” (I am not sure what they call themselves) have spent a lot of time trying to find common reasons for all kinds of things. For example:

• Common reasons why people do the wrong thing deliberately (peer pressure, not enough time in the day, rewarded for wrong behaviour etc),
• Common reasons people make mistakes (information overload, falling back on how the old system worked, lack of knowledge, lack of caring etc).

So there seems to be a whole field of study about how to make mistakes properly. I guess if you are going to do something you may as well do it properly … and since we make mistakes so often we may as well get really good a making them

But what is of more use to IT projects where people want to build good systems:

• If your system could kill people, probably get an expert in safety to test it.
• Even if your system won’t kill people … surely you want to anticipate the most obvious mistakes that users will almost certainly make.  So it follows that you would also want to at least do some simple testing to find out what happens when they do.

So I think it is fair to say that any team who want to produce good processes, good IT applications or good products will do some regular testing to see what happens when “the dumb users” make the same predictable mistakes we all tend to make.

So I would like assume every developer, BA and tester would do some simple exploratory testing  as they add features and I would assume that every team would do some regular regression testing as their system emerges.

But maybe that is the uncharted territory for human factor analysts – why do IT teams consistently make the same mistake of assuming the users will do exactly what they are meant to, when everybody knows they won’t?

Why would anyone spend$5m and 6 months of their life on a project and then only test how a user might actually use the system at the  very end of the project when there is no time left to modify the way the system behaves?

When you think about it though, shouldn’t we do a similar thing to define our project purpose when we start the project. I guess the answer is yes, but it could turn into a lot of work. So what we need is an easy, lazy approach.

This is what I think you can do on just about any project.

1. Define the problem you are meant to solve.
2. Define your scope.
3. Flavour to taste:
   1. Consider PAC to define the users
   2. Consider defining your product
   3. Consider adding an estimate
   4. Consider listing risks and other bad stuff

Paragraph one – choose one of these problem statements

• Problem statement from the book Fierce Conversations by Susan Scott (you should really read this book if you want to be both lazy and successful at the same time):
  • The problem we have is ….
  • It matters because ….
  • Ideally what we would like is …
  • So far we have done …..
  • Therefore, what we want to achieve next is ……
• Focussing question
  • How can [We] provide [Something] to [someone] so they can [do something or get some benefit or deal with some issue]
  • Eg – how can the IT team provide a database to Jenny so she can run her coffee club. OR how can Jenny provide a coffee club to her customers so they can buy coffee to take home
• The old fashioned story/legend format, which is the simplest you can use.
  • Currently [things are like this] but it sucks because [some annoyance] so we will [resolve the issue]
  • For example Jenny runs a coffee shop and she would like to sell take home coffee to customers so we want to create a coffee club to do this. Or Jenny has set up a coffee club for her customers but she is losing track of all the information she needs to track so we are gong to build her a database.

You can find a couple of other problem statements in a previous article.

Paragraph two – say what is in scope and what is not

No project ever fully solves a problem from every possible angle. If you build software, maybe you should train people and maintain the software. But even if you maintain the software for 5 years, at some point someone should replace the software with a better solution. But even if your project team agree to come back and replace the software, will they be the ones who pay for the carbon offset to make up for the extra greenhouse gases produced during the project lifecycle.

People generally understand that some things will be in scope and that others will be out of scope. Yet we often make wildly different assumptions about what actually goes into the scope.

So I think it is useful to define the following in at least some detail:

• This is in scope – deploying a database, maintaining it in production; and
• This is out of scope – ongoing training, maintaining the customer data.

This is often put into a table with the in-scope stuff on the left and the out-of-scope on the right.

But if you are a vendor or you are delivering to a demanding internal client you might choose to go one step further:

• The project team will deliver (in scope) – deploying a database
• The customer will – provide the team with access to users, supply coffee, provide testing
• The following is out of the scope of this project – ongoing training

Paragraph three onwards.

• Add constraints or priorities if you want to.  For example it needs to be in production before Friday next week (eek)
• Add PAC, Product statement or risks if you like
• Add an estimate of the effort involved if you want to
• Add the name of the sponsor, the provider and others if you really want to push the content.

If you want to you can even add a communication plan, a view of wether users will actually adopt the change we are implementing or a test strategy, but not all projects will need this much detail at the beginning.

A simple estimate

People often hesitate to provide estimates. But you can provide something as simple as this

We are 90% confident that the project will take between X and Y.  We could be more confident if [something]. For example:

• We are 90% confident the project will take between 50 and 150 years (let’s kill it now);
• We are 90% confident the project will take 3 developers between 3 and 5 days (let’s do it)’; or
• We are 90% confident the project will take between 3 weeks and 4 months. Hmm -
  • We can kill it if 3 weeks is too long;
  • We can get started if 4 months is tolerable and report back as we go. We will be more confident once we install the invergolator and get it running on windows 7; or
  • We could do more analysis.  This would add one week to the project but when we understand your needs better and review the deployment options for invergolators then we can provide a better estimate.  Let me know what you want us to do.

I have often found these estimates are enough to make informed decisions. But if you want to get far better at estimating then you should probably read “How to measure anything”.  That book even teaches you to estimate and understand the value of doing more estimates.  But the lazy approach is just to do what I did above.

What do you think?

It seems pretty basic doesn’t it.  So your project should probably have a more detailed description that everyone understands (the stakeholders and the team working on the project). But in practice I have often found a simple summary of the project is enough to make do.

In fact my quote for the day is :

Projects where everyone has the same simple understanding of the problem do better than those where some of the team have a really detailed understanding that is different to what the rest of the team understand.

I was shocked and horrified so I wandered off.

Soon after I spoke to some business analysts who were struggling with an agile project. They alleged that the agile approach they were using did not allow enough time to capture the requirements properly. But guess what, they broke down under interrogation and admitted that a large part of the problem they were having was that they did not really understand the product they were building.  Apparently the evil agile people had forced them to write stories about the system before they had time to understand it.

So I started wondering – is their any way you could actually understand a system or product before you built and tested it. This sounds like something you would want to do even if evil agile people want you to go really fast. In fact the faster you need to go in building something the less time you probably have to spend on wasted effort that is not related to what you are building.

I have already described PAC as a simple approach to user analysis but that describes what the users do and not what the system does for them.

So here is an old school approach that won’t take long.

Start by looking at the project charter/mandate and then ask people what the system does. Then do a little exploratory testing if the system already exists.

Now you are ready for a quick one hour workshop (or one hour of sitting in a cafe if you are on your own).

Step 1 – book a room for an hour or go to a cafe and order a skim milk cappuccino

If you have some friends who also want to understand the product then bring them along. Otherwise sneak off to a cafe on your own.  You will need a pen and paper and probably enough spare change to pay for you coffee.

Step 2 – create a one paragraph summary of what you think the product does

You can write you summary in any format you like, but I like the following approach which is loosely stolen from a book called “Stand back and deliver”. They use a similar approach to describe a team, but I find it works well enough for a product. It is also similar to the cereal box approach described in “Agile Project Management“.

The product provides [something] to [someone] so that they can [do something better]. Without it [someone] would have to [fall back]

If you are in a workshop setting, have each participant write this without speaking to the others. If you are on your own then I guess you should just write it out. But only allow 15 minutes for this exercise (plus 10 minutes to share and argue about the answers if in a workshop). You have a lot to do and you want to have time to drink our cappuccino.

Example

For example, lets say that Jenny owns the coffee shop you are sitting in. She wants to set up a new “wine and coffee” club so that people can learn more about their favourite beverages, order premium beverages to consume on site and even order bulk purchases for home.

Jenny decides that she wants to set up a new database system to keep track of the members of the club and if she can get the budget she will also add a web page with a blog and a social community for club members.

What does Jenny’s new database do and who is it for?  She could build it for the members or for herself … does it matter?

Jenny comes up with the following summary:

The coffee club database keeps track of coffee club membership information so that we can manage the coffee club. Without it we would need to have a manual system in place or forget the whole coffee club idea.

Optional step 2a – add some old school requirements (3-5)

With stories being all the rage these days, people sometimes often talk about the dark ages when we relied on requirements documents. But hidden in all those documents we could sometimes find some really clear and concise requirements that really did help us to build what people wanted.

First we used to list some functional requirements (what the system is meant to do) and then if you want to make good products you add some quality statements or “non-functional” requirements to clarify the style, performance and behaviour you want the system to have.

Here is my favourite format for functional requirements:

The [system or module] will [do something];

Or you can go all out and add two more two optional sections:

When [something] [the module] will [do something] if [some condition]

You can write millions of these one line requirements, but for the sake of understanding what the new product does you only really need about 3 – 5 lines. More than this will often cause people to start simply agreeing with them. Only listing the top 3 requirements will often cause people to have to think about what they mean.

Similarly, you can spend all day on this, but I would only allow 15 minutes, plus 10 minutes to share if you are in a workshop. Even 5 minutes might be enough.

So let’s see what Jenny came up with.

The coffee tracking system will store data on members of the coffee club

The coffee tracking system will store data on the coffees we have available and the promotions we are running at the moment with respect to those coffees

The coffee tracking database will store a history of promotions we have run and purchases users have made where privacy legislation and PCI compliance allow us to do so

The web page will provide static data on premium coffee and wine varieties and a blog for publishing regular coffee related news

When members join the online club the web page will provide access to a chat room where they can share coffee related views if the member has a valid login

Jenny has more work to do in order to fully scope and build the system.  But she now has a one page description that she can use to check whether she and her IT vendor (Charles who works for free coffees) understand the system.

Optional step 2b – add a quality statement or measure

Even if a system does what we want, it might not do it the way we want it.  For example

When we get home, my flatmate will cook dinner if I do the dishes after dinner

This sounds good – but I have had some flatmates who are great cooks and some who can make toast and coffee taste terrible.

So it is a good idea to define how you will know if the new system is good or not. Here is one (not too complicated approach):

The stakeholders will think [system or module] is great if [something)

The stakeholders will be really upset if [system or module does/does not … ] or [something happens]

Jenny might come up with:

I will be really annoyed if the system is hard for me and the barista to use

Customers will be upset if we leak their personal data or use it to spam them

I will be really happy if the system can run on my new galaxy tab as well as my netbook

Of course you are allowed to do better. A lot of people use iso9126 (a standard for software quality) to list the non-functional things the system should do (be reliable, be easy to learn etc). I often use FMEA (a standard way to be scared) or FAB (a sales technique I use for non-functional stuff).

Step 4 – Go and find out that you are wrong

You would think that it would be hard to get a one paragraph description of a system wrong.  But on almost every project I have tried this on I have found that people disagree on it.

So the final step is to share this paragraph with the project sponsor, the product owner or the project owner. They will correct your understanding and often surprise you.

If you have this simple summary, I can almost guarantee you will be able to do a lot of things much more easily – requirements are easier and so too is regression testing for example.

The only downside is that it can make it harder for developers and product owners who want to build without having to think about what they are doing. And of course you might need to spend up to an hour in a coffee shop to do the hard part of the work.