Are story points more trouble than they are worth?


#1

I recently posted a meme on twitter that read: “Agile…where the stories are made up and the points don’t matter.”

It was favorited 230 times and retweeted 186 times…which was about 1,000 times more than my normal posts. The post was a joke obviously, but I figure it must have resonated in the community.

My questions are this: given that many teams struggle with story points and that velocity is abused so often…are story points worth it? Or is it time to start pushing flow metrics from the beginning of projects?


#2

Great topic… This can be an interesting one. Sort of like asking a gear head what is the best brand of motor oil… For me I ask the question what are the points being used for? Reporting outward or just inward for the team… Its usually the combo. My next question would be what metrics and or are you measuring? Is it business value? Work done? Burn ups? ect. For me I like to use points to show a burn up to what this work is completing. So if there is a story and or epic or feature road map you can show progress or lack there using points. Could also accomplish this by saying stories done. Many ways to slice this question. Will be curious to read other opinions on this. The great no estimates debate…


#3

Literally just got back from presenting moving to flow based metrics. I am focusing on the Software Development Performance Index, that Rally and the Software Development Institute at Carnegie Mellon produced. It focus on having a balance between Productivity, Predictability, Responsiveness, and Quality. Organization can make trade off like pushing Productivity which usually increases Responsiveness, but you trade off Predictability and Quality.

The organization I am at now has a bunch of command and control metrics that have no diagnostically value what so ever. These metric are also used in teams performance reviews. So most are gamed and add no value.

The hope and pitch with my proposal it to use these diagnostical metrics and set bench marks like a Story Cycle time form in progress to accepted in 2 to 5 day. In hopes to correct some of the Fear and Anti-patterns the current metrics cause. I will most likely be submitting a proposal to the 2017 Scrum Gathering on this topic.

I am more than willing to go through this deck folks.


#4

I’m interested in this deck you speak of. I spend so much time trying to explain fairy points that I have begun to wonder about the cost/benefit of using them.


#5

Would love to see anything you have done in this area.


#6

How about this for an Agile Answer…It Depends:)

Estimates are a product of traditional project management and IT’s poor performance when it comes to delivery.

Shu Teams: Typically, the business/PO for these teams will want to see some form of estimation because they don’t trust the teams to deliver, and want to have something when it comes to the blame game. Story Points is a good tool to teach the team about relative estimating, and it helps drive conversations during planning poker sessions to allow the team to come to a common understanding of the story. If estimates are needed, we should derive them using a “value added” activity. Story Points and Planning Poker provides this…

Ha Teams: Trust is building between the PO/business and the team, however, they still need to something to hold on to. One approach is to break down all stories into the same size so stories delivered can be counted.

RI Teams: PO/Business no longer needs estimates because the team continuously delivers value every two weeks, and they just tell us what they want next. End Goal. #NoEstimates

The moral of this story is that until the teams earn the right to provide no estimates, there is value in using story point.

#NoEstimates is a perfect example of the adaptive nature of Agile and the influence of Lean.


#7

I can send you guys the deck and the spread sheet I am using. I am still looking at the bench marks and what is going to be the most value. Like a tring to keep a user story in 2 to 5 days of cycle time or may be a threshold for planned and unplanned (prod bugs)work. In build you fixit world. Would love some thoughts on that. Give me to the beginning of October and we could do an interactive meeting. I should stop jacking @andybacon thread.


#8

Excellent, excellent, excellent points @mccallam2! @Scrummando make sure you credit us when you put that in your talk :slight_smile:


#10

@mccallam2 for the win. Damn dude. Solid points.


#11

@troy I’ve seen those statistics before too. As much I believe Schwaber has that data, there are too many variables in my mind that would influence the outcome. Like size of teams, experience of members, build and deployment capabilities - just to name a few. In the last year, I have gone from mocking #NoEstimates people to almost joining their dark, twisted crusade. Fortune 25 vs. Non funded start ups will certainly have different perspectives - and yes, agnostic of @mccallam2 's Shu, Ha, Ri argument. Though I do think that was a solid answer, I would love to explore company size, structure, and goals and smash that up against @Scrummando 's flow based metrics research. Awesome thread so far…


#13

Yes, story points can be a pain. Will they have an effect on the velocity of the team? No, at least as long as the team is trustworthy. Do they at least to create trust? No, they are always a matter of mistrust.

Instead of investing into estimates based on story points, I have started to invest into trust. And this really pays off. No more blaming, full focus on efficiency and effectiveness.

If you need to report progress, use flow metrics as suggested. If you need to plan, make a best guess or ask the team to tell about ambitions and add some buffer on top. This tends to be better than a forecast based on story points and velocity in more than 80% of all cases.


#14

I’m dredging this one back up from the depths today.

Given my choices, I’ve historically been a no estimates advocate. I unfortunately usually get outvoted on this one and I’ve accepted that for the most part and I’ve seen some pretty good teams find comfort in planning with points. The reason for my excavation is I’ve started with a new role in a different part of the organization, and they are awfully concerned about points, and by proxy, team capacity.

So it gets me thinking - what is the purpose of using these points? We always caution that points != hours/days, but really, if we ask what the team’s velocity is that’s exactly what we’re doing - how much work can the team complete in the next 2 weeks. I spent some time this week digging into the data available in Jira, and generally, I’m seeing a terrible correlation between cycle time and the point size - as in the trend line value for even the best teams I’ve seen is < 10% - and that’s after removing the outliers.

So then… what is the point here? I’ve always maintained that if the team is always working on the most important things at any given time, decoupling dependencies and building good work flow, the rest will take care of itself.


#15

I pretty much feel like Joshua Kerievsky does in “Stop Using Story Points”.

Story points were introduced in the hope that they would help people worry less about estimates, but people still get hung up about them. For the most part, I prefer to encourage teams to put their mental effort into splitting stories smaller instead of estimating them in points. But I also think it’s fine to give a time range for bigger things (“it’ll take us 2-3 months to achieve business goal X”). An explicit acknowledgement of uncertainty is really important, and I think points were supposed to give that but largely failed.

I’ve also found James Shore’s perspective in his “Estimates or No Estimates?” talk intriguing. His conclusion is that teams just starting out with agile development won’t be able to give accurate estimates, so they shouldn’t try; slightly more advanced teams can give accurate estimates, and maybe should if they need to build trust; but the most advanced teams are focused on delivering value continuously, and estimates just get in their way.


#16

That article sums up my thoughts much more eloquently than I have written, thanks for sharing.


#17

I guess my view is different because a chunk of my career has been working in software situations where the product is somewhat new (1st gen, not yet production quality, within the first 3-5 years, and still gaining revenue streams… meaning not yet profitable). One can easily argue that it’s a bad idea to start a product that will take 5 years to get positive revenue, (and it’s a case to coach on not understanding MVP or proper product management), but these are typical environments I keep finding myself drawn to helping in small/mid-size companies.

In these environments, you have technical teams without a long history on an established tech stack. It’s almost impossible to form this immature team and size all stories a similar size.

AND, the business doesn’t like the “it will take 3 years” answer to their claimed “MVP scope”. Story points become the collaborative negotiation point to determine whether you are truly building MVP, and whether adding more staff is an option to cut the timeline down.

When companies are working with rounds of investment and a board that has very specific date milestones (yearly tradeshows that must NOT be missed to succeed)… you have to estimate and project somehow. Flow and trust is a great goal, but if you run out of funding for this project in X months, you have to be able to find out quickly if you are anywhere near the mark. That’s what story point projections have always provided me (80% of the value and accuracy with 20% of the effort).

I’ve also never participated in gaming points and their projections, which seems to be the basis of Joshua Kerievsky’s article linked above.

All that being said, hours estimation, or estimates on tasks under stories make me gag… as they always smell of waste or micro-management to me.


#18

Nobody can provide accurate estimates in the domains we operate in: complicated and complex.

All estimates are a guess.

However, imo, that does not make estimating not useful. It’s the estimating, not the estimate.

During “estimating sessions” I encourage quick relative sizing of stories (Bockman method) - yes using story point buckets (powers of 2) - and focus the conversations not around accuracy or precision but instead around readiness, sliceability for the “larger ones”, and the act of sizing helps that.

While we do use story points, there’s no implication that they correspond to time. They give a young team a chance at sanity, that’s all. “Rolling average of three sprints = 100 points, why are we committing to 200 this iteration?!”

In all my years, I’ve had the privilege only once of working on a team where we got to the point of simply counting stories themselves - “We can take on 50 stories per iteration” - and that took a helluva lot of work, not so much on the dev-team role, but on the product ownership side of things.

Hourly estimates on subtasks… gasp… horror show down that lane.