Suggestions for Assessing a Coach's Performance

#1

I recently took over a newly-established Agile CoE and have a mixed bag of coaches. Some are FTEs, others are contractors. I am looking at the best way to evaluate the coaches’ performance because I cannot use the teams’ maturity and progress.

Here is the rub - the organization is relatively new to Agile, so we have the usual challenges - teams are still being established for projects then disbanded, people (ie, Product Owners and Scrum Masters) were placed in those roles by their management teams with little to no guidance, so overall there are a number of challeneges out of the coaches direct influence that are impacting the teams’ maturity and growth.

The coaches are primarily working with the teams, I do not have coaches that really coach at the leadership/executive level…also part of my challenge.

Any ideas, recommendations would be greatly appreciated.

Thanks!

#2

Simple NPS survey?

How likely is it that you would recommend [this agile coach] to a friend or colleague?

#3

I agree with Andy… A simple NPS/eNPS might help. But, coaching from the top is an absolute must have. I have found that teams really do want to be self-organized. It’s always about the managers. I would start small… a pilot team. Don’t just get buy in from managers. Make sure that they are ACTIVELY part of the transformation because they need to transform as well. One thing I have done in the past if a book study 2x a week over lunch with managers… Mike Cohn’s book “Succeeding with Agile” is an excellent primer for discussion. It’s worked for me in the past.

#4

@andycleff and @davidnovick, thanks for the excellent options. I will definitely go the NPS route, as well as check out the book. Thanks!

#5

I have a contrarian view.

I don’t see how the team coaches can do their jobs if the people who are supposed to do the work don’t know what they’re supposed to do and the teams are being disbanded. A highly effective coach might anger people by asking the team to work in a way that the team doesn’t understand. NPS means the team likes the coach, not that the coach is effective.

I would start with management. The teams might need to learn some agile basics. They probably need to learn about technical excellence. And, none of that is going to occur if the managers don’t change.

I would take the coaches and create a CoP for the coaches. I would ask them to coach managers. I’m sure the managers don’t work in teams. What can a coach do for that?

And, if the organization wants Scrum-as-idea, sure, keep doing what you’re doing. If the organization wants to use agile approaches, the coaches are coaching the wrong people.

#6

Always an interesting and engaging perspective, @johannarothman !

A highly effective coach might anger people by asking the team to work in a way that the team doesn’t understand.

Agreed. Comes with the territory.

NPS means the team likes the coach, not that the coach is effective.

Disagree somewhat. Depends on when the NPS survey is conducted, how it is rolled out, and what is meant by “recommend”

Personally, I wouldn’t recommend someone that I simply liked. I could dislike them and if they were highly effective, I still might recommend them. “Coach X is a total hardass, calls it like it is, and boy that hurt - when the problem was ‘me’. It wasn’t fun, but I learned a lot about my blind spots. Highly recommended if you’re up for the pain.”

the coaches are coaching the wrong people
Yes!!! I forget where I first read this:

“When a flower doesn’t bloom, you fix the environment in which it grows, not the flower.” – Alexander Den Jeijer

And of course there’s always Demming:

“Every system is perfectly designed to get the results it gets.” Deming

Coach to change the system… wherever the most significant problems exist… and I think @johannarothman is pointing to the likely suspects :slight_smile:

#7

I am also leading a newly-established agile coaching team, so I can relate to some of these challenges. For me, the most important thing right now is to establish a clear charter for the team. It’s my job to engage with the executive level as a coach myself and get the necessary agreements about how the rest of the coaches are going to work with managers and teams. I’m asking each coach to build relationships with a few senior managers to set the stage for future engagements. And I’m getting to know the coaches as well as I can in short order, so as to leverage their strengths and to support their own professional development.

For more on chartering, I recommend Liftoff (2nd Ed.): Start and Sustain Successful Agile Teams by Diana Larsen and Ainsley Nies. The book is about the overall process of launching a team, and chapters 4 to 7 are specifically about team charters. It’s mostly for software development teams, naturally, but there are some examples of service-providing team charters as well, which contain some inspiration for a coaching team charter (e.g. the last example in Appendix 1, on page 127).

#8

I suspect that drafting the charter won’t be a problem. You’ll get to some reasonable agreement.

I suspect that getting sponsorship from management (not just agreement) might be more of a problem. Hmm, I seem to be more curmudgeonly these days. I might need to write a blog post :slight_smile:

#9

As a coach working in an AC4E that is growing fast this discussion brings to mind a thought “if you want to know how I’ll behave, tell me how you’ll measure me”.

Which then brings my thoughts to this question - “What outcomes do you expect your coaches to enable?”, which will then inform how to assess performance.

For delivery teams, the things I value and coach to improve are:

  1. The team values reflection, learning, and continuous improvement
  2. The team acts and behaves like a true team - people that trust each other!
  3. Outputs and outcomes are high quality - with quality being context-specific
  4. A context-specific balance between predictability and adaptability
  5. The team is working at a sustainable pace and satisfied with their work environment

These can form the foundation of the outcomes your coaches work towards with their teams and thus how their performance is assessed.

The Team-centric stuff is necessary but not sufficient. @johannarothman makes a solid point re: changing the environment the team is operating within is crucial. If your coaches are focused on and enabled for team-centric concerns, their outcomes will be constrained by the environment and system they are operating within.

Where you go with this depends on where you focus your coaches’ energy.

#10

IMHO, if you’re assessing the coaches’ ‘performance’, you have already lost.

It’s a fruitless endeavour. One team’s “A+” coach is another team’s “GET THAT PERSON OUT OF HERE” coach.

At risk of sounding really hippy-dippy, all that effort is best spent on truly knowing your coaches, so that you can find them a situation where they can succeed, and place them in that situation. And if you don’t have any of those situations, you should let them go.

This is probably true for most professions, but it’s especially true for coaches.

#11

A fellow coach once said their indicator of performance was how many people of a particular level in the hierarchy they pissed off in a week.

Less than a x VPs, the coach was not doing a good job!

#12

Hey! Partner the coach with the team’s line manager, Director, or whomever is providing the vision and identify goals that need to be met. Have the coach tailor their interactions to focus on those goals and objectives. Check periodically, rinse, and repeat. It’s all about the agile approach to coaching. We eat our own dogfodd right?

Hope all is well!

Steve