Are story points more trouble than they are worth?

#1

I recently posted a meme on twitter that read: "Agile…where the stories are made up and the points don’t matter."

It was favorited 230 times and retweeted 186 times…which was about 1,000 times more than my normal posts. The post was a joke obviously, but I figure it must have resonated in the community.

My questions are this: given that many teams struggle with story points and that velocity is abused so often…are story points worth it? Or is it time to start pushing flow metrics from the beginning of projects?

#2

Great topic… This can be an interesting one. Sort of like asking a gear head what is the best brand of motor oil… For me I ask the question what are the points being used for? Reporting outward or just inward for the team… Its usually the combo. My next question would be what metrics and or are you measuring? Is it business value? Work done? Burn ups? ect. For me I like to use points to show a burn up to what this work is completing. So if there is a story and or epic or feature road map you can show progress or lack there using points. Could also accomplish this by saying stories done. Many ways to slice this question. Will be curious to read other opinions on this. The great no estimates debate…

#3

Literally just got back from presenting moving to flow based metrics. I am focusing on the Software Development Performance Index, that Rally and the Software Development Institute at Carnegie Mellon produced. It focus on having a balance between Productivity, Predictability, Responsiveness, and Quality. Organization can make trade off like pushing Productivity which usually increases Responsiveness, but you trade off Predictability and Quality.

The organization I am at now has a bunch of command and control metrics that have no diagnostically value what so ever. These metric are also used in teams performance reviews. So most are gamed and add no value.

The hope and pitch with my proposal it to use these diagnostical metrics and set bench marks like a Story Cycle time form in progress to accepted in 2 to 5 day. In hopes to correct some of the Fear and Anti-patterns the current metrics cause. I will most likely be submitting a proposal to the 2017 Scrum Gathering on this topic.

I am more than willing to go through this deck folks.

#4

I’m interested in this deck you speak of. I spend so much time trying to explain fairy points that I have begun to wonder about the cost/benefit of using them.

#5

Would love to see anything you have done in this area.

#6

How about this for an Agile Answer…It Depends:)

Estimates are a product of traditional project management and IT’s poor performance when it comes to delivery.

Shu Teams: Typically, the business/PO for these teams will want to see some form of estimation because they don’t trust the teams to deliver, and want to have something when it comes to the blame game. Story Points is a good tool to teach the team about relative estimating, and it helps drive conversations during planning poker sessions to allow the team to come to a common understanding of the story. If estimates are needed, we should derive them using a “value added” activity. Story Points and Planning Poker provides this…

Ha Teams: Trust is building between the PO/business and the team, however, they still need to something to hold on to. One approach is to break down all stories into the same size so stories delivered can be counted.

RI Teams: PO/Business no longer needs estimates because the team continuously delivers value every two weeks, and they just tell us what they want next. End Goal. #NoEstimates

The moral of this story is that until the teams earn the right to provide no estimates, there is value in using story point.

#NoEstimates is a perfect example of the adaptive nature of Agile and the influence of Lean.

#7

I can send you guys the deck and the spread sheet I am using. I am still looking at the bench marks and what is going to be the most value. Like a tring to keep a user story in 2 to 5 days of cycle time or may be a threshold for planned and unplanned (prod bugs)work. In build you fixit world. Would love some thoughts on that. Give me to the beginning of October and we could do an interactive meeting. I should stop jacking @andybacon thread.

#8

Excellent, excellent, excellent points @mccallam2! @Scrummando make sure you credit us when you put that in your talk :slight_smile:

#10

@mccallam2 for the win. Damn dude. Solid points.

#11

@troy I’ve seen those statistics before too. As much I believe Schwaber has that data, there are too many variables in my mind that would influence the outcome. Like size of teams, experience of members, build and deployment capabilities - just to name a few. In the last year, I have gone from mocking #NoEstimates people to almost joining their dark, twisted crusade. Fortune 25 vs. Non funded start ups will certainly have different perspectives - and yes, agnostic of @mccallam2 's Shu, Ha, Ri argument. Though I do think that was a solid answer, I would love to explore company size, structure, and goals and smash that up against @Scrummando 's flow based metrics research. Awesome thread so far…

#13

Yes, story points can be a pain. Will they have an effect on the velocity of the team? No, at least as long as the team is trustworthy. Do they at least to create trust? No, they are always a matter of mistrust.

Instead of investing into estimates based on story points, I have started to invest into trust. And this really pays off. No more blaming, full focus on efficiency and effectiveness.

If you need to report progress, use flow metrics as suggested. If you need to plan, make a best guess or ask the team to tell about ambitions and add some buffer on top. This tends to be better than a forecast based on story points and velocity in more than 80% of all cases.

#14

I’m dredging this one back up from the depths today.

Given my choices, I’ve historically been a no estimates advocate. I unfortunately usually get outvoted on this one and I’ve accepted that for the most part and I’ve seen some pretty good teams find comfort in planning with points. The reason for my excavation is I’ve started with a new role in a different part of the organization, and they are awfully concerned about points, and by proxy, team capacity.

So it gets me thinking - what is the purpose of using these points? We always caution that points != hours/days, but really, if we ask what the team’s velocity is that’s exactly what we’re doing - how much work can the team complete in the next 2 weeks. I spent some time this week digging into the data available in Jira, and generally, I’m seeing a terrible correlation between cycle time and the point size - as in the trend line value for even the best teams I’ve seen is < 10% - and that’s after removing the outliers.

So then… what is the point here? I’ve always maintained that if the team is always working on the most important things at any given time, decoupling dependencies and building good work flow, the rest will take care of itself.

#15

I pretty much feel like Joshua Kerievsky does in “Stop Using Story Points”.

Story points were introduced in the hope that they would help people worry less about estimates, but people still get hung up about them. For the most part, I prefer to encourage teams to put their mental effort into splitting stories smaller instead of estimating them in points. But I also think it’s fine to give a time range for bigger things (“it’ll take us 2-3 months to achieve business goal X”). An explicit acknowledgement of uncertainty is really important, and I think points were supposed to give that but largely failed.

I’ve also found James Shore’s perspective in his “Estimates or No Estimates?” talk intriguing. His conclusion is that teams just starting out with agile development won’t be able to give accurate estimates, so they shouldn’t try; slightly more advanced teams can give accurate estimates, and maybe should if they need to build trust; but the most advanced teams are focused on delivering value continuously, and estimates just get in their way.

#16

That article sums up my thoughts much more eloquently than I have written, thanks for sharing.

#17

I guess my view is different because a chunk of my career has been working in software situations where the product is somewhat new (1st gen, not yet production quality, within the first 3-5 years, and still gaining revenue streams… meaning not yet profitable). One can easily argue that it’s a bad idea to start a product that will take 5 years to get positive revenue, (and it’s a case to coach on not understanding MVP or proper product management), but these are typical environments I keep finding myself drawn to helping in small/mid-size companies.

In these environments, you have technical teams without a long history on an established tech stack. It’s almost impossible to form this immature team and size all stories a similar size.

AND, the business doesn’t like the “it will take 3 years” answer to their claimed “MVP scope”. Story points become the collaborative negotiation point to determine whether you are truly building MVP, and whether adding more staff is an option to cut the timeline down.

When companies are working with rounds of investment and a board that has very specific date milestones (yearly tradeshows that must NOT be missed to succeed)… you have to estimate and project somehow. Flow and trust is a great goal, but if you run out of funding for this project in X months, you have to be able to find out quickly if you are anywhere near the mark. That’s what story point projections have always provided me (80% of the value and accuracy with 20% of the effort).

I’ve also never participated in gaming points and their projections, which seems to be the basis of Joshua Kerievsky’s article linked above.

All that being said, hours estimation, or estimates on tasks under stories make me gag… as they always smell of waste or micro-management to me.

#18

Nobody can provide accurate estimates in the domains we operate in: complicated and complex.

All estimates are a guess.

However, imo, that does not make estimating not useful. It’s the estimating, not the estimate.

During “estimating sessions” I encourage quick relative sizing of stories (Bockman method) - yes using story point buckets (powers of 2) - and focus the conversations not around accuracy or precision but instead around readiness, sliceability for the “larger ones”, and the act of sizing helps that.

While we do use story points, there’s no implication that they correspond to time. They give a young team a chance at sanity, that’s all. “Rolling average of three sprints = 100 points, why are we committing to 200 this iteration?!”

In all my years, I’ve had the privilege only once of working on a team where we got to the point of simply counting stories themselves - “We can take on 50 stories per iteration” - and that took a helluva lot of work, not so much on the dev-team role, but on the product ownership side of things.

Hourly estimates on subtasks… gasp… horror show down that lane.

#19

@andycleff - Accurate Estimate is an oxymoron !

I have been using story points to relative size stories to bring some clarity to what the team can actually do and as a planning guideline and forecast for the PO’s just as @andycleff and @kschlabach have said.

But, behold, some of the are horror stories that I have heard from others include story points being used as a metric to compare between teams, a measure that the team has to “increase” each sprint and in some cases as a performance metric between individuals in a team. Often, this is being done by someone in a powerful management position and the team is silently having to accept to do this.

Great discussion so far, everytime I speak to a group about story points, the opinion is divided, people take sides (and almost get into fights :smile:) and eventually there is no conclusion and the answer finally is “it depends”.

#20

I’m currently writing a blog post on my thoughts around story points, so I’m hoping this post will help me as much as those reading it!

I aim for my teams to use story points for two reasons:

  • To see if a story is too big to bring into a sprint
  • To see if the team have a satisfactory amount of alignment about the complexity involved

We do record the story points, but this is the last time we use them. We use card counting to see how much was achieved and for capacity planning. When it comes to planning I like to average over the last six sprints, divide by the amount of sprints that I’ve averaged, and then take 60-70% of that. This gives high confidence that we can deliver at least that, and most of the time we exceed that expectation.

@andycleff I’m currently in the position where I’ve converted two teams in the programme I’m working on to this method. I’m still convincing the teams that it works in their favour, yet management is fully sold on the method!

Ron Jeffries on #NoEstimates
#21

Still convincing the teams

Management is fully sold

Funny, I commonly have the reciprocal problem :slight_smile:

Have the teams been able to articulate their concerns?
What are you trying to show the teams the value of your idea?
How will you know if this experiment is a success or a failure?
What is the next experiment you might try?

#22

Oh me too! This is the first time that I’ve been successful in trying this out. It’s been a complete success. We do quarterly capacity planning, and the team delivered only one fewer stories than predicted.

I think the concern of the teams are that it won’t satisfy management. We’ve told them management is happy, but I don’t think they will be convinced until management have been happy for at least a year. Another reason is that they don’t understand how we can card count when stories aren’t the same size. We do get them within a range, and as we’re working with averages it works out. Overall, I think that they’ve had so many people tell them that to be agile you need to be calculating your velocity in story points that they think other ways are wrong.

I have on their boards my predictions for the quarter we’re working through sprint by sprint, and also how many cards they were predicted to complete and how many they completed. I think this has made a difference, but I don’t have any ideas of what to do next.

Unfortunately, I don’t think I’ll be in my current role much longer (I’m a contractor). If I do stay, I’d like to look more at the longer term planning that we attempt to do in this programme.