Favourite reading on agile project management

I thought I would just put up a quick post to point people to my favourite reading on agile and related project management topics.

Read More »

Is reactive planning a good approach to running projects? Or is waterfall a better approach?

I often tell my wife that I am a lucky project manager – that I can often just let things play out let them come together. 

My wife claims it is not project management at all if there is no planning and no management.  She is right of course, but there is such a thing as “reactive project management” which pushes the boundaries a little.

Reactive project management means starting the project with a rough idea of what to do but no clear plan.  Then you form a plan as events unfold.

This is a common approach in television shows where high drama results from watching the characters almost die each episode.  So, should we plan projects the way they do in TV shows or is that a little dodgy?

Forming a plan as you go can be a valid approach to projects in two situations that I can see:

  1. Crisis management where the original plan just went out the window.  In  these cases it is necessary to keep moving forward while coming up with a new plan.  I have used this approach in disaster recovery situations and it is both more fun than watching TV and very effective.  But these projects only last 24-36 hours, and after that I think it is time to down tools and do some formal planning if things are still dragging on.
  2. Pure research where you don’t know what opportunities or risks exist in a situation and you want to experiment for a while to see what you can learn.  For example if you want to know how Ipads work with your internal IT systems you might give some ipads to some developers and tell them to play with the new toys for a couple of weeks.  This can be quite effective as long as you (and your paying customer) are clear that all you are likely to get from the exercise is some more knowledge rather than any clear deliverables.
  3. Where situations are so unheard of and people so clever that the best thing to do is to train them well and then parachute them in to work out what to do.  This is similar to the situation above but aims to achieve an actual outcome.  In this case though you would have to assume both a high risk of failure and an almost innevitable series of hits, misses and rework.
  4. When things are trivial or simple so there is no need to plan.  Of course these situations rarely constitute projects.

So if we define “agile projects” for a team as “coming up with a plan as we go” then I think the approach will have only limited use. Yet on many projects, if you ask the team what the plan is then they will look confused or tell you that they don’t really have time to plan.

This is good news if you want to be involved in high drama (like tv shows) or in rescuing the project down the track.  But generally it spells trouble, as predicted by the old waterfall saying that “failing to plan is the same as planning to fail”. 

In these cases I would say that predictive planning (waterfall) is generally a better approach than what the team is calling “agile”.

So I think I have established that agile is generally better than bureaucracy (see my previous article) and waterfall is generally better than letting a plan emerge as you go.

Is bureaucratic project management a good idea? Or is agile better?

Is bureacratic project management a valid way to run projects? Yes – rarely but sometimes this is a good approach. Generally though, bureacracy is bad and using agile approaches is better.

So that was a short article 🙂 

But if you have time for a biased and semi-accurate history or bureacratic project management then read on.  This is a long article though so grab a tea or coffee and sit down comfortably while you read it (or skip to the next article). 

Otherwise feel free to click on this link to see a shorter article on agile versus waterfall: https://kingsinsight.com/2010/10/03/is-agile-better-than-waterfall/

Bureacratic project management is a suprisingly common approach to projects.  It is the process of outsourcing thinking to a methodology. 

At one point in history I suspect many projects were guru led – ie one great thinker (or at least one fairly organised person) start to tell people what to do and things worked better when they did. 

But over time this led to issues when more than one great thinker existed in the team and they told people to do different things.  It also led to issues for the “one great thinker” because people kept coming up to him (or her) and asking what to do, or else they did their own thing and tripped over people who were trying to do what the guru said.

So people started to agree that it would be better to have a set of procedures and to follow them.  Then they could assign specific rights and obligations to each other and get on with their work without tripping over each other. 

Of course people would encounter situations which the procedures did not address and so they would need to escalate to a certified guru (or “supervisor” as they were often called).  These gurus also had a set of procedures for handling exceptions and they also had more experience than the “front-line non-gurus”.

Over time more and more things got put into procedures and this was great, except that there always seemed to be more exceptions and more new situations to document.

In more recent times, we started to document some procedures as law and require compliance or punishment would occur.  But within organisations (and societies) it turned out that not everything was a predictable procedure.

In particular, large new projects required planning and analysis because they combined uncertainty with complexity.  So the existing procedures no longer worked.

Some well meaning people with a lot of experience (scientists, engineers and adminstrators) gave up on documenting everything and instead documented a standard approach to the questions you should ask when running (what were at the time) large projects.  They then broke the problem of delivering projects down into stages so that people could focus on what they were doing without losing sight of the big picture.

While many approaches now exist, most of them come down to these stages

  • Work out what the problem is;
  • Investigate possible solutions;
  • Decide on one or more solutions and plan how to implement them;
  • Implement the solution; and
  • Hand the solution over to someone to look after.

Since most large problems were engineering ones the general approach tended to adopt what became a sound set of techniques for building solid structures.  So “Implementing the solution” was often standardised as:

  • Analyse the problem and design a solution in some detail;
  • Once you are confident of the design, plan how to implement it in some detail;
  • Build the solution;
  • Test the solution; and finally
  • Deploy the solution.

Cool – this approach started working really well so some of the people using it started to document what they did and share it with others. 

Unfortunately this is where things went a little wrong.  Rather than designing a new approach to every problem, people started applying the same (documented) approach to the problem. 

This is NOT what the authors of PMBOK, RUPP, SCRUM, PRINCE 2 or other approaches recommended.  They were assuming that the process for implementing their suggested approaches would be:

  1. Learn the approach we recommend.
  2. Engage brain.
  3. Understand the problem you are solving and adapt our approach to solving the problem.
  4. IF you are unsure, research the approach more, ask for help or wing it (ie do what you think is right)
  5. Take accountability for the outcome.

Unfortunately what I now refer to as “Bureaucratic” project management is to adopt the following process:

  1. Learn the new approach.
  2. Ensure every document, meeting and stagegate is completed in accordance with the instructions.
  3. Assume that if all the steps are followed in the right order then the project will be a success.

This approach works well if:

  • The documents are like voodoo artefacts and they capture all risks and expectations within themselves and then magically alter reality.  This means that if you alter the document then you will alter reality in the same way as manipulating a voodoo doll will impact real people;
  • Other people are getting the project done for you while you focus on the documents, and the customer is happy to pay for both the documentation process and the actual project (where this is true, perhaps we should offer leather bound copies of all project artefacts and hold all project ceremonies in expensive restaurants);
  • The completion of the documents (and meetings) is more important (or at least as important) as delivering the solution the project is designed to produce.

So lets look at whether these circumstances ever apply to a project and thus whether bureacratic project approaches are valid.

Even voodoo practitioners will probably tell you that the documents used in most projects are not valid voodoo talismans.  They hold no power over reality and thus this arguement is invalid. 

Of course it might be possible to have the books and ceremonies in your project blessed and this may, if magic works, impact your success.  But I have not attempted this approach and cannot provide valid advice on its prognosis for success.

Would the customer ever be happy to pay for both the project and a beautiful set of documents.  The answer is yes.  I don’t just mean that they want status reports that aid decisions or that they want supporting material to handover to their support teams either.  I actually mean that they will pay for ceremonies that make them seem important and that they will pay for documents that they can show their friends (and bosses) to make it look like a project is complex, successful and generally very impressive to behold.

Should you provide these documents and ceremony when they provide little or no value beyond this?  Generally no.  But if you are happy to be paid to do this then the choice is yours.  But remember this work is not helping to deliver the project and so you will need to allocate a separate budget to doing that at the same time, don’t be fooled into thinking the documents are the project or that the stakeholders will thank you when reality looks far less impressive than what your documents and ceremonies predicted.

In fact if you buy into this approach then why not consider going all the way – hire a group of actors and set designers to build a whole (fake) copy of your project.  Fund this as the shop front and impress people with what is going on while a separate team do the real work and deliver a solution without being distracted by all the acting, ceremonies and documents.

So far its all a bit negative though isn’t it?  What about our final circumstance – the documents are as important as the project.  Is this ever true.

Well, cynically, this is the case if you are being paid per document and don’t care much about the end result beyond this.  Outside of the world of dodgy consultants and disgruntled employees this is rarely (if ever) the case.

But it might also apply when confronted with compliance demands from a client. For example if you need to submit tenders in a set format or with answers to specific questions (eg a “request for proposal or RFP”).  In this case you will care about the overall project but the production of the document needed to continue is a project in itself.  Where this is the case, I would recommend bureacratic compliance with requirements but also remember that the document is a separate project on its own and then a constraint on other aspects of the project, but not a tool for delivery.

Similarly, some clients will demand that you produce reports and other materials that do not aid in the delivery of a project.  Where this is the case I think the professional approach is to tell the client that this adds no value to the work you are doing and is being produced purely for their purposes.  Again though, this is no longer an approach to the project but a separate deliverable. 

So the final circumstance creates an additional deliverable, an additional cost and potentially constraints or issues on the project.  But this is not expediting the delivery of a quality product, so it should not be viewed as a project approach as such.

So, is adaptive (or predictive) project management generally better than this?  YES – almost always.

If you are afflicted by bureacratic project management then adopting a new approach will generally be an improvement.

Is agile better than waterfall?

Many IT teams are moving from “waterfall to agile based approaches” in their projects.

So clearly a lot of people think that agile approaches are better than waterfall approaches.  But are they really?  And if they are then how are they better?

One argument I often hear in favour of agile approaches is this one:

Is there any point in having a 100 page requirements document that nobody reads? No? Then waterfall is stupid and we need to use agile.

I often start with this statement when running project management courses because it is a great way to start a debate.  But it is not a sound argument.  In fact it is very similar to saying:

Is there any point in driving a truck to work if it won’t fit into the garage?  No? Then driving a truck to work is stupid and you should ride a horse.

There is actually no logical connection between the question and the conclusion.

So let’s take a step back and define what we are talking about when we look at “agile versus waterfall”.

What different approaches exist for running projects?

A lot of work has been done to examine the different methodologies used to run projects.  If you are interested a good summary of current thinking then I recommend reading “Manage IT” by Johanna Rothman: It provides a detailed description of different approaches, together with some pragmatic advice on how to apply them to IT projects.

But I am going to completely disregard all the sound research and advice from expert practitioners such as Johanna.  Instead I have invented my own taxonomy of project approaches.

So, based onmy not-very extensive research and consultation with no experts from any field,  I am going to claim:

  • That there are four generic approaches to running projects; and
  • That each generic approach can be broken into two specific approaches.

So these are the approaches I believe can be used to successfully deliver a project:


Cause based projects – Sharing a common belief but not dictating how each group or individual accomplishes their contribution.

Guru (or expert) based projects – Running the project on the basis of the expertise and judgement of a key individual.

Repeatable process: Compliance – Determining the standards and procedures to be used and then executing each project according to the agreed standards.

Repeatable process: Continuous improvement – Determining the standards and procedures to be used and using these, but (really) challenging and improving the standards each time they are used.

Bureaucratic waterfall – Completing documents for the sake of completing a template.

Predictive planning – Using sound engineering principles to run the project which involves performing detailed analysis at each step to predict what needs to happen in future stages.

Reactive planning – Starting the project without a plan (or only a limited one) and then creating a plan as events unfold.  Can also be described as “thinking on your feet”.

Adaptive planning – Using the scientific method to run projects.  Set a hypothesis at each stage and determine how that hypothesis can be disproven.  Compared to predictive planning this approach replaces detailed analysis with the testing of the hypothesis.

Which approach is the best?

I will explain each approach in more detail in a separate post (or posts) but generally people usually ask the wrong question when arguing in favour of waterfall or agile:

  • A popular (but biased) arguement for waterfall is to compare predictive project management to reactive project management and to then ask the audience which is better;
  • A popular (but biased) arguement for agile is to compare adaptive project management to bureacratic project management and to ask which is better.

A better arguement is to start with the problem the audience want to tackle and then ask these questions:

  • What problem are we solving?
  • What are the risks we face in implementing a solution?
  • Is the problem small enough to
    • Simply use our standard process (compliance or continuous improvement)? 
    • Simply leave it to the team or an individual to implement (Thus relying on the team to understand our cause and create their own solution.  To rely on the individual we need to clarify their authority and expertise and then tell people to follow them).
  • If the initiative is bigger, riskier and more complex and we are involving enough people for long enough to establish a formal project then, are we better off using predicive planning, adaptive planning or both?

Whatever the answer is – that is the best approach to this project.  Again I will expand on my views on this in some other articles.  But it turns out that no approach yet discovered is universally the best.

Of course those who know me will note I have a strong bias for cause led approaches to large initiatives, continuous imporvement for ongoing services and tasks, reactive planning for my own life and adaptive planning for projects.