Measuring teams - how, what, when, why?

#1

Continuing the discussion from Flow, Decoupling Cadences, and Fixed-Length Sprints:

Can I measure the performance of a (cross-functional) product team?

On the first sight it seems to be easy to measure the progress, throughput, issues and incidents from the plans and the systems we are running. Drawback: we are relying on predictions and take assumptions, that the systems we are working with are stable (no side effects, no disruptions, no trends, no changes in setups).

On the second sight it turns out that product development is not an engineering task only, but requires a lot of thinking, experimenting, creativity, and collaboration. This brings with it the idea of measuring how well a team is doing - a health check.

There are many ways to do health checks within a team. From my point of view the daily check-in is already a start, but it does not have any metrics.

@andycleff provides a great summary of various team health and morale checks in his blog post, which gives a great start.

Anyhow, what is it? How do you perform health checks and above of all, what are you looking for? What is the purpose?

I for myself have developed a team health check a while ago with the purpose of helping teams to find topics for the retrospectives. It worked pretty well, but is nothing which could have been tracked. It is moreover an exercise to encourage reflection within a complete team:

https://coalition.agileuprising.com/uploads/default/original/1X/18d8c5965f65b4143ef750a5ae92c9e8acc260f6.png

I for myself do hard in measuring teams today, as I fear being too narrow in what I am looking at. I want my teams to find out for themselves how good they are doing their job and how they can become better. I can support in measuring, but not measure to support.

How do you think about it?

#2

Thanks for branching this topic, @hdietrich

I want my teams to find out for themselves how good they are doing their job and how they can become better. I can support in measuring, but not measure to support.

I think that ^ is a wonderfully healthy way to look at things. And would hope more organizations would adopt such an enlightened viewpoint.

Unfortunately, I see many places where trust is very low. Teams are overly concerned that any metrics they use and make visible will be used as weapons by others, not as ways of finding out for themselves how their experiments are going…

I’ve been working on a talk proposal for the 2018 conferences “42 others things besides velocity for teams to consider…” and am gathering a smorgasbord of options… with Team Health and Wellbeing being one of the most powerful leading indicators, imo.

The other “main dishes” come from Jason Tice @theagilefactor - and overlap with your four areas of inquiry:

  • Process Health Metrics
  • Release Metrics
  • Product Development Metrics
  • Technical / Code Metrics
  • People/Team: The Human Elements

I suggest that teams choose what’s important to them “now” - a handful of things - and get some visualization going of the unordered domains they dance in. And then they can run some experiments and probe/act, sense, and respond.

An early blog post on the topic of metrics as a holistic approach:

And first pass presentation from 2017 on “how, what, when and why”…: https://drive.google.com/file/d/19ZCd-PY4bL76QPB1thvJbgGM3CQinMA6/view?usp=sharing

…that is the basis for my 2018 session proposal…

@Scrummando also did a talk on metrics at Agile DC 2017: http://agileuprising.libsyn.com/5-metrics-to-create-safety-and-high-performing-teams

#3

I applaud you for trying this and helping me think about possibilities. I worry about measuring happiness.

I can only speak for myself. I prefer being “satisfied” with my work, which for me is a function of:

  • interesting work (so I can learn)
  • a collaborative environment (so I can work with others)
  • knowing I am delivering something people need (so I know my work has value)
  • earning the respect of my peers (so I know I am contributing to the greater whole)

Again, for me, happiness is an outcome of these pieces.

I am probably not the norm. I wonder what is the norm for people, or if there is one?

#4

I agree that “measuring happiness” is prob the wrong mindset.

Measuring something like health and well-being might be closer to the meaning I’m looking for.

#5

100% - this addresses one of the fundamental issues I experience again and again and again:

If people want to improve a business, they start by looking at others, but not at themselves. Therefore metrics get misused for the sake of highlighting deficits and advice on how to improve.

Whenever everyone of us l start by looking at himself, we would understand, which value a measurement can bring to us when avoiding bias and heuristics in our observations. Everyone simply would make better decisions on themselves first before turning over to others. That would be real empowerment.

#6

The analogy of a doctor might help in here. Healthy people do not attend to a doctor. But when you feel sick, the doctor uses several “metrics” to find out what is wrong with you. Still it is about you to rest or take some medicine.

(… a good doctor is also looking for the soul!)

#7

I been knees deep in this topic for 18 months with the emergence of Lean Agile Intelligence, the industry’s first outcome driven customizable Agile and Lean Assessment platform, so I have a bit of an opinion:)

I also start with a very simple health check, but it is in the form of an assessment of course:). Very similar to the approach of @andycleff and @johannarothman , the questions focus on the team‘s well being and culture. It is pulse check of specific topics such as to safety, happiness, autonomy, self-organization and sense of purpose to name a few. From experience these are all prerequisites for a high performing team; so if we have problems in any of these areas we should be focusing on them first.

The assessment format helps drive the conversation, and gives people more context on what it means to be happy, safe, and empowered in the workplace. It is also an easy way to identify improvement areas to help teams get to the next level. It is for the team, and the team only, and votes are autonomous.

Once teams evolve into more of a norming stage, they should focus on measuring fluency and metrics in four different areas:

  1. Doing it Fast: Time to Market & Responsiveness Practices & Metrics
  2. Doing it Right: Customer Satisfaction & Practices & Metrics
  3. Doing it On Time: Predictability Practices and Metrics
  4. Keep Doing It: Employee Satisfaction and Innovation Practices and Metrics

This model is based on Larry Maccherone’s SDPI (Software Development Performance Index) that was created as a result of a study of over 10K agile teams sourced from Rally. You will find metrics and practices in these different areas will “pull” on one another, forcing the team to decide what is really important giving their context and desired business outcomes.

@Scrummando and I have taken the model a step further and injected measuring fluency on agile practices that lead to these outcomes, through assessments of course. Fluency is a leading indicator to better outcomes and is a great way to identify improvement areas that will truly help you move the needle! Shameless plug: if you get a chance, visit and sign up for Lean Agile Intelligence, to see how our platform supports this model. It is free for four months and we would love your feedback. https://www.leanagileintelligence.com/

#8

I think we can expand the doctor analogy…

I’m encouraged to have “well visits” - to get my blood pressure checked, my cholesterol, and as I age, a whole host of other pokes and prods and probes.

The idea being these metrics are all leading indicators of my future feeling well…

I envision the Team Level Metrics in the same light: what are leading indicators, what are lagging indicators…

And I prefer the former over the latter. And I still believe in measuring team health, wellbeing, culture (mood, morale) can point us at least to “look here” and ask questions about what is going on.

@mccallam2 - good on ya’ for plugging LAI!

#9

Nice objection, having “well visits” :smile: - I didn’t think about it.

Let me jump on that train and ask for the benchmarks and conclusions. Many of the benchmarks are set by the pharmaceutical industry and people take actions, which do not make them really happy (diet, pills, sports, etc.), just to opt-in for a longer life.

There is a thin line, which is getting back to personal appreciation. What do I like for myself and what not? What is it worth it from my point of view?

P.S. I proposed the use of LAI to one of my teams a while ago, but they did not feel really happy with the suggestions of the tool. It was driving conversation, but more about the need for norms, as they felt like it is taking away flexibility (agility) from them. The barrier has been more of a psychological denial rather than doubts about the validity of the proposals. My conclusion was, that they have been just missing the discussion to come up with the proposals on themselves, which seemed to be leading to the idea that they have been infantilized.

#10

Yikes. Treat people like children… and you’ll get messy diapers…

Could you measure “safety” - team member silence vs voice? Part of your instrumentation towards creating an environment for the teams to be able to say “Stop with the parental shit” -

Could you measure also effective decision making? moving to the lowest possible level of hierarchy delegation / decision making / consensus ?

#11

Love it!

#12

A big belated thank you – terrific question and awesome replies, just starting to do some work on this in relation to an operating model change. If there’s been any additional thinking, or published blogs / talks on this would love to see. And once I have some idea of how we’ll be going forward on this I’ll update!