Product Owner and Scrum Master Metrics

Now that Agile has gone mainstream, I often find that people are concerned about the performance of the members of an agile team.

In particular people seem interested in how we measure the Product Owner and Team Facilitator (Scrum Master, Agile Coach, Celebrity Post-it note Artiste etc).

When working through this, I came up with some “useful” metrics, but this still seemed to leave a gap … which led me to wondering if it is the metrics we need, or an understanding of what we mean by “self-organizing teams, served by scrum servants and product guides.”

In fact one interesting conversation began when one of the crew asked if there was such a thing as a “senior Scrum Master.” I told them no – in fact I said that scrum masters become more junior and more like team servants until they become a team lacky, or retire from work to become an agile coach.

This seemed to get people upset. Especially because the agile coaches told me that they were not retired – they were working really hard, but just weren’t sure what it was that they were working on. Anyway, that is a different topic – we agreed that agile coaches are really there to help people learn to be more effective as a team and everyone was happy on that point.

For now though, I thought I would share my own logic on metrics for PO and Scrum Master types in order to see if it leads to agreement or grave concerns.

I will also let you be the judge of whether this is just a fun article or if it is hinting at a gap that we can fill in the understanding people have of the roles.

Inspect and adapt and other good things

In agile teams we love to inspect things and improve our performance. Even the humble user story can have multiple metrics inflicted on it – cycle time, control charts, aging, velocity and maybe more.

It follows then that we would also apply this to our people – we can inspect and adapt in order to help them improve.  

There is one problem though – they do not operate in isolation. In fact the the best evidence of a good PO and a good scrum master is that the team is able to quickly and easily add value without the intervention of the PO and scrum master. So team effectiveness without intervention is probably the best metric.

In the early days the team needs guidance, but as time goes on, presumably, the PO and the Scrum Master are creating greater clarity and learning … which means the team become clear on what to focus their energy on, how to measure success, how to work together and how to manage their own work. 

A rating for evolving irrelevance

If the goal is to build a self-managed team that is independent and effective, then it follows that a good metric would be to assess how long it is until the PO and scrum master are completely irrelevant. Ultimately, when they have become completely irrelevant, both the PO and the Scrum Master have been of exceptional service, but if they are not yet irrelevant then perhaps we could estimate the days left until they expect to become irrelevant. I have included such a metric here:

But what about the rest of the team

Agile work is team bases, so we cannot expect the PO and Scrum Master to make themselves irrelevant without help. So maybe we can look at some leading indicators across the team, to see if they are supporting the emerging irrelevance.

Collaboration driven irrelevance

The good news is that I have done some research into this very topic – maybe the Scrum Master is needed because the team need help with collaboration.  This means that they are still relevant as long as the team hopes to either improve their collaboration or to at least have some help facilitating it. 

This in turn means that the scrum master cannot become completely irrelevant until each team member is completely collaborative.

So now we have a metrics hierarchy – If I rate each crew member on collaborative they are then I can move them all to 100% collaborative, at which point the Scrum Master is completely irrelevant.

To drive the collaboration score for each team member we will need to tackle the following drivers:

• Building trust with the rest of the team so the rest of the team are happy to collaborate with them
• Learning to trust the rest of the team, so that they are happy to collaborate without fear
• Helping the team member to get better at the core skills of listening, talking, giving feedback and engaging in healthy conflict so that collaboration is actually effective
• Removing impediments to effective collaboration, such as distractions, misunderstandings, a lack of clarity of the work and so forth
• Building formal and informal collaboration channels and then helping the team member master these, so they can collaborate easily
• Sadly, the final driver is that the team member must want to collaborate, this is something only they can shift, not the scrum master. So if people don’t want to collaborate, we have a problem – If the scrum master is irrelevant but people don’t want to collaborate then we need to stop doing all the agile practices, since they just become overhead.

So first the Scrum Master must confirm that people want to collaborate. If not then the will not be effective. But once people are willing to try, the Scrum Master chips away at building the factors that lead to their irrelevance.

A minimum viable metric

Measuring days to irrelevance is good, but seems difficult. Ultimately you can adopt the following formula, but people always seems dubious. The predicted days to irrelevance is actually the greater of (90 days or the number of days the person has already been in their role). So if someone is new, we should see them becoming irrelevant in about 90 days. But if someone has been in the role 45 days, then we still predict that they have 90 days to go. If they have been in the role for 180 days, then we can expect them to stick around for another 180 days.

But most people don’t believe that is a real metric, so you can try this one:

Or if you want more detailed metrics – you can replace these two questions with the set of drivers we apply to team members:

Irrelevance based on autonomy

So measuring the increasing irrelevance of the Scrum Master is easy, but what about the PO?

It’s a similar journey – the PO exists because we do not trust that the team would work on the right things if they were not there, or we don’t trust that the organisation would give the team permission to do what they want.

In other words we are not fully ready to make the team truly autonomous.

So perhaps we should measure whether the team members can act with autonomy or not. In theory autonomy is good, but not if we don’t trust the team … or more politely we don’t yet think they are ready for full autonomy.

The key driver here is alignment, followed by understanding and finally competence.

If the PO does not believe that the team are aligned to the goal, then the PO cannot step away. Once the PO is clear that that the whole team is aligned to the right goal, then the PO starts to become less relevant.

OK – the first metric for a good PO is to measure the clarity of the goal and the commitment of the team to achieving it.

So far so good. But some teams appear incapable of breaking their own work down or understanding how to achieve the goal without a PO to guide them. In this case, the PO, or a mini-me PO is still needed.

So the second driver of team autonomy is the ability to recognize, prioritise and focus on valuable work.

Finally, and this is controversial, the PO will probably not be willing to move away if he or she doubts the competence of the team. In theory, the team might be clear on the vision, clear on the priority and completely inept at doing the work.

I have been asked what the PO should do if they do not believe the team is competent at achieving the goal … but the answer is a bit sad. Either you are the wrong PO, you need to change the mission to one which includes time for the team to learn, or you have the wrong team.

So the final driver for the PO is to measure whether the team are confident and trusted to do the work well. If they are not, then the PO must either seek resources that are needed, or prioritise team learning and growth.

OK then, just like the Scrum Master needs to have a team that is willing to interact, the PO needs a team who are willing to become competent.

Once the team is willing and able to become competent, then the PO can begin to become irrelevant. After that the metrics are similar to the Scrum Master.

If we do not want to measure days or percentage irrelevance then we can use the following minimum viable metrics:

So – will this work

I have spoken to people about these metrics and some are quite happy, even though the metrics seem informal.

Others seem to want more robust metrics of team alignment and collaborations but I am not sure that is necessary. Others want more robust metrics on the value the team is creating (revenue, joy, etc), which seems fair enough. I think we should focus on that more than the individual performance.

Some people don’t like the idea of the PO and Scrum Master becoming irrelevant, they want to believe that the team will always need them. The aim then will be to have single point sensitive PO/Scrum Masters who are micro managing the team.

In that case I guess you can just reverse the metrics and measure team dependency and helplessness. The metrics will still work, I’m just not sure if you have the right role descriptions.

The chief cause of distress seems to be that many people do not agree that the performance of the PO and scrum master is derived from team performance. They want to be able to say that the PO and scrum master can do good work as individuals, even if the team completely fails to add value and completely fails to collaborate.

I think the whole point of both roles is to make the team (or group of teams) more effective and more self-managing. But if you want to measure something else then maybe measure “number of meetings attended per week” or “delightfulness of the velocity chart” or something.

More seriously though – I think it is possible to measure the performance of the PO and the Scrum Master and I see value in doing so. But the challenge is to shift the mindset from personal outcomes to “supporting the team to deliver outcomes” and then to shift again from “managing the team to deliver outcomes” to “teaching or enabling the team to achieve outcomes.”

I think that is the sticking point. It’s not the metrics that are hard, it is seeking agreement on the point of the role and the value that it should create.

In my view the point of both roles is to unleash the potential of the people in the team and then get out of the way – but I am not sure if you agree.