Stop Using Story Points (2012)
Sprints, standups and story points have come to symbolize Agile methods much like burgers, fries and cola symbolize fast food.
Ready for your Agile Happy Meal?
I hope not.
Like researchers of fast food, we now know that the Agile Happy Meal contains unnatural ingredients that decrease agility and cause process indigestion.
In 2007, a series of experiments led my colleagues and me to increase our agility by dropping story points and velocity calculations.
Those same experiments led us to replace fixed-length sprints with a flow-based workflow, and move from standup meetings to frequent team huddles.
Our process today looks nothing like a by-the-book, mainstream Agile method largely because we actively look for process waste and experiment to discover better ways of working.
In this blog, I’ll explain why we dropped story points and velocity calculations and what you can do to work successfully without them.
My Early Days with Story Points
I started using story points on an Extreme Programming team in a San Francisco startup in 1999.
I loved how story points helped identify a team’s work capacity.
That capacity, calculated as velocity (number of story points completed per timebox), helped people on teams (especially product managers, product owners or internal customers) learn to work within their means.
If the work to be done in a timebox didn’t add up to the team’s velocity, it was time for some critical thinking and decision making about must-haves vs. nice-to-haves.
Story points and velocity helped teams save time by identifying essential functionality and removing anything unnecessary (or “maximizing the work not done”, an often-repeated mantra).
I used story points and velocity for another 8 years, both on internal projects and on client projects around the globe.
During that time, I noticed many misunderstandings and misuses of story points and velocity calculations.
At first, I interpreted these problems as a sign that we needed to do a better job of educating people.
But as the years passed, it became clear that story points and velocity calculations contain deep flaws.
Irrational Story Point Inflation
One of first big problems I noticed happened in a large, famous corporation headquartered near Tampa, Florida.
We first got to know this company in 2002, when a Director (I’ll call him Jim) came out to San Francisco to experience our intensive Extreme Programming Workshop.
Jim loved what he experienced and wanted to immediately bring it back to his teams in Florida.
We went on to have a 4.5 year relationship with hundreds of people within Jim’s company, including 2 of the 3 major corporate divisions.
For the first team, Jim bought a complete agile transition package, including our readiness assessment, training and coaching services.
He took our advice and designed a fantastic open-space for the 25-person team (big by agile standards at the time).
He had his entire team study and discuss Patrick Lencioni’s Five Dysfunctions of a Team prior to us working with them.
When we landed in warm, sunny Tampa, we met one of the best-prepared teams we had ever encountered.
Fast forward 2 years and this team had shipped a great deal of excellent software ahead of schedule, received healthy bonuses for their work and won the parent company’s prestigious process improvement award.
Executives who visited the company would routinely tour this team’s open workspace and learn about their process.
We were proud of this team and completely unprepared for what was about to happen to them.
One day in 2004 Jim exhorted the team to go faster.
He used the term “sprint” even though he wasn’t familiar with Scrum.
This team had an average velocity around 52 points per iteration.
Their velocity would flunctuate by a few points, but generally remained steady.
Yet just weeks after Jim asked the team to “sprint”, the team’s velocity jumped up into the high 80s!
I asked someone what happened.
She looked at me funny and said “These days around here if you sneeze, you get a story point.”
I shook my head, amazed at how a mature agile team, a team that had been assessed, trained and coached by myself and two excellent Industrial Logic coaches, could so suddenly inflate their story point estimates to appear like they were going faster.
My confidence in story points and velocity calculations began to erode after that experience.
Yet perhaps the problem had been due to insufficient education?
Rather than abandoning story points in 2004, my colleagues and I redoubled our efforts to better educate our clients, especially management, about the misuses of story points.
Sacrificing Quality For Predictability
Technical practices like test-driven development and refactoring are often the first things to be dropped when someone is trying to “make their estimate.”
We see this behavior emerge from the outset, starting in training: a team that completes all work at the end of a timebox (say, two hours) often does so by sacrificing quality.
For years, we addressed this common problem by helping students experience what it was like to deliver on commitments while also producing quality code.
We explained how great teams ultimately learned to maintain technical excellence while also sustaining a consistent velocity (the same number of story points completed per sprint).
Yet the fact is, such teams are rare and their consistency is often fleeting as their team size changes or other pressures take hold.
For too many years, I thought it was important to maintain a consistent velocity, since it was supposed to be helpful in planning and predicting when a product/system could be completed.
Yet I’ve seen too many teams sacrifice quality and face compounding technical debt solely to make or exceed their estimates and keep their burndown charts looking pretty.
Comparing Teams By Points
Over the years I’ve heard several managers at different companies ask, “Why is team X getting 24 story points done per sprint, while team Y is only getting 12 story points done? Those teams are roughly the same size, so why the discrepancy?”
Instead of treating velocity as a team-specific, capacity indicator, such managers fall into the trap of thinking of velocity as a performance measurement (as Jim Highsmith described in Velocity Is Killing Agility!).
We can educate people to not compare teams by their velocities, yet it is a real problem that so many people naturally reason that such comparisons are legitimate.
Starting around 2005, I tried to deal with this problem by having teams name their story points after favorite types of fruit.
One team woud measure velocity in oranges, another would measure it in apples, another in kiwis, etc.
Then, when management invariably tried to compare the velocities of teams, I would tell them that “You can’t compare apples to oranges.”
While this approach helped to point out the fruitlessness (pun intended) of comparing team velocities, it led me on a path of doing more and more work to stop people from misusing story points rather than shift to a simpler and less error-prone method.
Story Point Confusion: Are We NUTs?
Many justify the use of story points because they say that humans aren’t good at estimating using actual time.
They say that story points make us better at estimating because we’re estimating the size of work, rather than the time it takes to complete it.
In fact, humans aren’t very good at estimating in general, regardless of what measure is used.
As I will show later, teams get better estimates when they re-estimate frequently.
Story points just further muddle the issue.
While we grow up estimating using time (e.g. “When will you be ready to go?” or “How long before you arrive home?”), the same cannot be said of story points.
To use them, we must learn:
- what they are,
- how to use them,
- the many ways to not misuse them.
A typical education in story points contains a good deal of friction.
Newbies are confused and uncomfortable estimating with story points and frequently compensate by consistently translating story points back to actual time estimates.
Well-meaning agile experts do their best to educate the uninitiated.
They explain how story points are like t-shirts sizes or Fibonacci sequences.
In 2005, one of our customers found story points to be so confusing that he renamed them NUTs (Nebulous Units of Time).
Misunderstandings and misuses of story points generally multiply over time, especially as a process scales from team to department to enterprise.
Fixing those misunderstandings and misuses costs time and energy.
Story points and velocity calculations are unnatural techniques that unnecessarily confuse teams at the beginning of their agile journey.
Busy-Ness Accounting
I visited a company the other day that had giant kanban boards filled with technical tasks in various stages of completion.
I couldn’t tell from these physical boards if the teams were working on valuable user stories.
I couldn’t see progress on release plans and looking at their web-based Agile planning tool didn’t provide much help either.
Instead, the tool was ready to show me pretty velocity charts and report on how many hours tasks took to complete.
So many Agile shops seem to be stuck in this mode of tracking busy work.
My colleague Tim Ottinger calls it “busy-ness accounting.”
Breaking down a user story into tasks is fine to understand what will be involved in completing it, but spending loads of time estimating tasks
and tracking the tasks on agile planning boards/tools is a lot of wasted energy and a relic of waterfall-based planning schemes.
Being agile means moving with quick, easy grace.
That means releasing useful, quality software frequently.
It doesn’t mean being busy accounting for how every hour is spent doing technical tasks.
A Simpler Way
In 2001, I remember being struck by a simple planning approach mentioned by Don Wells, an early Extreme Programmer who worked on the Chrysler Comprehensive Compensation System project (the birthplace of XP).
Don said (in this email from the Extreme Programming email list):
We have been counting items done. Each week we just choose the most important items and sign up for them up to the number from last week. It turns out that we get about the same number of them done regardless of estimated effort. We have 1 week iterations so we tend to break things down a bit at the iteration planning meeting.
Perhaps the effect is that we have learned how to break things down to the right size. I don’t know yet, but the point is we get about 8 things done each week, no estimation required.
While I wasn’t ready at the time to try Don’s simple planning approach, he had whet my appetite for a simpler planning formula.
Experimenting With Alternatives
By the Spring of 2007, my colleagues and I had seen enough problems associated with story points and velocity calculations that we began experimenting with simpler solutions.
Our first experiment within Industrial Logic failed badly.
We tracked real-time hours instead of story points, but we were essentially doing the same thing with hours as we had done with story points: carefully accounting for estimated hours, completed hours and a new budget of hours available for the next fixed-length timebox.
This project lasted approximately 6 months and afterwards, more than half of the people on the project said they disliked using real-time estimates with hours.
That Summer I attended the Agile2007 conference in Washington, DC and heard David Anderson speak about his Kanban method during an open space, birds-of-a-feather session.
I loved what David had described and it inspired me to continue searching for a process that better fit my company’s needs and culture.
In the Fall of 2007, we stopped using story points or hours and started working in what I called (at the time) “variable length iterations.”
No longer happy to work in fixed-length timeboxes, we would simply pick a small amount of important work to do, complete that work, ship it and repeat.
In general, we didn’t care if it took 1 day or 4 days to get the work done.
Our focus was on shipping, not working in a fixed timebox or tracking the number of story points completed.
Using the new process, we shipped (on average) 1-2 times per week.
Our agility had increased by removing once-sacred pieces from our process.
We were now even better at delivering on the promise of the Agile Manifesto’s first principle:
Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
If we had a deadline, we would make it by either getting everything done or finding work that could be safely deferred until a later release
Our planning sessions were much shorter and we judged ourselves on whether we had shipped valuable software, not whether we had maintained a consistent velocity.
I learned that what I had called “variable-length iterations” was actually more like a flow-based workflow in lean development.
And in lean terms, I started describing story points as waste, like the empty calories in cola.
Gut Feel
In 2008, Industrial Logic was asked by a Silicon Valley startup (now owned by I.B.M.) to help one team inside their company become agile.
One of our coaches worked with that team for several weeks, teaching and coaching them in traditional agile practices, including the use of story points.
At the time, we were not yet comfortable teaching our story point-free approach to our customers.
A few months after this first project completed, the company asked if we could now help them spread agile across the whole company.
We worked out a transition plan which included having me interview the first team that had already gone through the process change.
When I walked into that team’s open workspace, I saw the walls covered with posters showing burn-down charts, release plans, iteration plans and charts tracking the team’s velocity over time.
I sat down with a woman who had a reputation for being the company’s best product manager.
After a few minutes of talking, I pointed to some of the posters on the walls and asked how things were going with planning and estimation.
For a brief moment she looked at me funny.
Then she said that everything was just fine, that they were tracking velocity with story points and using their velocity to schedule just the right amount of work for their iterations and releases.
Something about her expression suggested that something was amiss.
“Are you sure?” I asked.
She paused for several seconds, then said, “Do you want to know the real answer?”
I said yes.
She then said “You see those posters with the velocity numbers? We made all of those up because we knew you were coming to visit us and we wanted you to say nice things about us to our management.”
“Really?!?”, I said, rather surprised. “So what do you actually do?”
She explained that she and her team simply used gut feel to choose the right amount of work for a 2 week iteration or 2 month release.
If they needed to adjust the plan, they did so, always keeping stakeholders informed.
She said that this gut feel approach to estimation and planning had been working just fine and that they had no need for story points or velocity calculations.
I complimented her on the simplicity of her approach and assured her that what she was doing was perfectly agile, that using story points wasn’t necessary to be agile.
And then I explained how Industrial Logic had stopped using story points for our own product development a year earlier.
Periodic Release Planning
Many of us estimate in order to know when we can release software to our users and what it will roughly cost.
At the beginning of a project, most release plans and dates are a nice fantasy.
You could spend days or weeks attempting to get more accurate estimates, yet I find it is better to just make rough estimates for each user story based on 10-15 minutes of analysis and discussion per story.
To get higher value for less cost, I coach teams to do Bargain Hunting.
To get more accurate estimates, I coach teams to periodically (say, every 2 weeks) re-estimate the user stories on their release plans.
As time marches on, teams that regularly re-estimate their release plans get faster at it and provide time-based estimates that become increasingly accurate as the release date approaches.
That gives management what they needed all along: a decent way to predict the scope, schedule and cost of a release.
With this approach, there is no gaming of velocity numbers and the focus repeatedly returns to scoping discussions and what will/won’t be released.
I coach teams to write user stories for their releases and estimate them using “team weeks”: the amount of time it will take the whole team to complete the work.
It doesn’t matter if only 2 people on a team of 10 can do the work: just provide the estimate as either .5, 1, 1.5 or 2 team weeks
If the estimate for a user story exceeds 2 team weeks, I ask the team to split the story so that it can be estimated at or below 2 team weeks.
To better manage risk, it’s useful to combine evolutionary design with release planning.
Your first embryonic version of the product/system should include high value features and address your highest risks.
That release (which could take 1-3 months to produce) may be an internal milestone rather than something released to customers.
The next release will add further sophistication to the emerging software by adding functionality to existing features or producing new features.
And so it continues until you are ready to release to your customers.
All along the way, a team must make sure that the number of estimated team weeks on their release plan fits with the number of calendar weeks left for the release.
If there is a discrepancy, it’s time to re-scope, re-think and re-plan the release so it fits with the team’s work capacity.
I’ve found that this style of planning is simpler and yields better results than estimating with story points and feeling bad that you didn’t hit your estimated velocity for a given sprint.
Periodic release planning helps people manage risk by repeatedly focusing on what valuable functionality can be delivered.
What’s the Point of Story Points?
Story points and velocity calculations are now seen as defacto parts of Agile, legitimized by popular books, embedded in planning tools, verified in certifications and widely practiced in the industry.
Yet nearly all of the veteran agile practitioners I know haven’t used story points or velocity calculations in years.
In the early days of Agile we thought there was a point to story points.
Yet we were careful to not let our ideas and practices ossify: to become set in a rigidly conventional pattern.
Today, we have simpler, less defective agile estimation and planning approaches.
Some now do what Don Wells once recommended in 2001: to work on the same number of similarly-sized stories each sprint.
Others work without sprints in a flow-based model that includes periodic release planning.
While there will never be one right way to be agile about estimation and planning, you would do well to avoid yesterday’s outmoded and highly defective techniques.
Or in other words, be careful of what’s inside your Agile Happy Meal.
Accounting For Busyness Agility Busyness Busyness Accounting Continuous Delivery Estimation Release Planning Story Point Confusion Story Point Inflation Story Points Variable Length Iterations Velocity