Scrum Points: Why Story Points Are Better Than Hours

In Scrum, many people who have managed projects with hours have a hard time understanding why story points are better.

Scrum Points: Why Story Points Are Better Than Hours | OpenView Labs

They have failed to understand some fundamental data that has been published for over 20 years in the industry literature.

First, let’s look at the latest data on project failures. [Last year], failure rates increased for IT projects during the disruption of the global financial system. A Standish group survey showed that 68% of projects failed to meet original estimates. In fact, Forrester Group research shows this as well:
Common Project Management Metrics Doom IT Departments to Failure

Looking for more Scrum tips from Jeff?

Check out our exclusive videos below, and be sure to view our Scrum Resource Guide for even more information.

None of this information is new. The venture capitalists I work with say they’ve never seen a correct Gantt Chart in a board meeting. When they dig down into the problem, they say none of their management teams knew the velocity of their teams before they implemented Scrum. Not knowing the velocity of team production is the root cause of 100% failure of [accurate release plans] in their board meetings.

You would think this data would cause people to change their behavior, but many companies seem to prefer to continue to fail and be acquired or go bankrupt rather than improve their project management techniques. The research clearly shows that humans are not good at estimating hours and practical experience repeatedly confirms the research.

The management metric for project delivery needs to be a unit of production. Production is the precondition to revenue, and companies say they are in business to grow revenue and margins (even though in project planning they often do the opposite). At least a venture capital group is clear that it is all about the money, and money comes from velocity of production combined with quality of the product. Hours are expense and should be reduced or eliminated whenever possible.

The best data on individual developer performance comes from Yale University and has been reported previously on my blog. The best developer on a project takes one hour to complete a task while the worst developer takes 10 hours (within a project) or 25 hours (across projects). For teams, the difference is an order of magnitude greater. Larry Putnam’s published data shows that an hour for the most productive team turns into 2,000 hours for the least productive team.

Hours completed tell the Product Owner nothing about how many features he can ship and when he can ship them.

The metric of importance is the number of story points the team can deliver per unit of calendar time. The points per sprint is the velocity. Therefore we estimate everything in points for the Product Owner so that he can create a release roadmap based on team velocity and adjust the plan if velocity changes.

The way we do story point estimation is better than hourly estimates as it is more accurate and has less variation. A CMMI Level 5 company determined that story point estimation cuts estimation time by 80%, allowing teams to do more estimation and tracking than a typical waterfall team. A telecom company noticed that estimated story points with Planning Poker was 48 times faster than waterfall estimation practices in the company and gave as good or better estimates.

Story points are therefore faster, better, and cheaper than hours, and the highest performing teams completely abandon any hourly estimation as they view it as waste that just slows them down.

(Editor’s note: This article originally appeared on Jeff Sutherland’s Scrum Log.)

Dr. Jeff Sutherland is the co-founder of the Scrum development process and a Senior Advisor to OpenView Venture Partners. He is a renowned trainer and speaker focused on helping businesses increase productivity through the power of Scrum. For more from Jeff, visit his Scrum Log blog site.

(Editor’s note: This article originally appeared on Jeff Sutherland’s Scrum Log.)

photo by: marfis75

Share Your Thoughts

  • Qwertij

    Nice of you have the luxury of not needing to quote a project first (being a vendor) or if you do not need to create a business case on what a certain set of functionality will cost the business. How would you go about that BEFORE you know the velocity of the team?

  • Jeff Sutherland

    Go to scrum.jeffsutherland.com and click on Jeff Sutherland’s Papers and download “Scrum and CMMI” papers. These detail how a CMMI Level 5 organization uses Scrum for fixed price projects (virtually 100% of their business) and cuts 80% of planning costs, 50% of project costs, and 40% of defects on average. This company can do a perfect waterfall project but these are bid at twice the cost of a Scrum project.

    Now they have historical data (as do even startups after a few iterations) and they use it. For a startup with no Scrum experience run three two week sprints and you have a reasonable velocity established. Alternatively you can make a guess and try to hoodwink your investors. Many of them are still clueless (with the exception of OpenView).

    If you are not a product company you can do the same thing. If management will not invest a few weeks to get good data, they deserve to get the crappy results they have come to expect. I’ve never talked to a management team that liked the results of what they were getting from IT unless they have implemented an effective Scrum and were actively participating in companywide improvement of the results by removing impediments.

    • Utter77

      Hours vs Points, Cost vs Velocity

      I totaly get the idea of using points instead of hours to estimate your work. BUT! I thinkt that whenever a person asks of “when” you have to answer in hours instead of points. Some would say that you can answer “when” by knowing your teams velocity and say, “we can deliver XXX points in one sprint.” But you only know your velocity by knowing how many points can be delivered in how many days. With that said, each point has a time-value. Which is also preffered by in that hyped book “…in the trenches” quote: …corresponds roughly to ideal man days…
      And any employer (at least in Sweden) would say that an ideal man day is 8 hours.

      Sure you can get a good velocity estimation for the team in 3-4 sprints but you can get a much better and faster velocity estimation for the team if you use hours and a factor.

      Let me demonstrate:
      Team: 4 developers (8 hours a day)Sprintlength: 10 days (10*4*8 = 320 manhours / sprint)
      Case 1 (Story points)The team has a bunch of items in their backlog. They estimate the effort in points. Before the first sprint they have no idea of the teams velocity but they estimate it to 30 points and plans the first sprint according to that. They fail the first sprint and only after 4 sprints have they got a team-velocity estimation of 24 points per sprint. They know they use 320 manhours per sprint so each point can at this time be valued about 13 hours. This will be lower when they realize all the time meetings and other stuff takes. So in maybe 8 sprints do they know that one point can be valued about 10 hours.Time to get velocity estimation: 8 sprints.
      Case 2 (hours)Same three items but they use hours. They estimate the items in real hours. They know they have 320 manhours per sprint and they use a factor of 75% = 240.They plan items to fill the 240 hours and they succeeds with the first sprint.The rest of the time they finetune the factor and sees what they can do to increese it.
      Time to get a velocity estimation: 0.
      Both these cases results in a team-velocity and both velocitys have a mesurable time-value.
      But when you initially use point, you waste a good deal of time..

  • Anonymous

    Story points are a lot like function points that have been used in software development in the past.  In the end you need to be able to equate you measurement criteria (hours or story points) to a dollar amount.  You really don’t seem to address that.  As Quertij says, it is a luxury to not have to be able to estimate jobs.  

    What it comes down to is that scrum can be used for internal groups that are on a steady funding.  By not having requirements for a system, you get what you got at a point in time.  In addition, scrum addresses the production aspect of software, not the design trade-offs, or the measurement of effectiveness of the final product.  When you talk about the venture capital group looking at volume of production, you are mixing metaphors.  If I am producing a product, yes, the velocity of production is important.  But that applies to products that are produced, as in an assembly line.  When you talk engineering, there is another whole dynamic.  

    My experience working at and with a number of CMMI level 5 companies is that we could estimate jobs very quickly and accurately in a very short period of time.  The first of these I was with, in the mid 1980’s had extensive information on hours, lines of code (and other engineering artifacts) that could be used to estimate a job.  The trick was not lines of code or function points, it was the understanding of the thing to be designed, understanding the historical information, and then applying that knowledge to the actual data.  This worked quite well.  

    What I have seen when organizations move to agile (or try to) is that they apply a strong measurement discipline.  In many cases they had lost the discipline and had no real methodology.  That is why they failed before and why they improved.  It was not so much the particular methodology, as imposing a methodology.  In some big organizations, there are groups that have gone agile, but still have 10% – 15% of their projects being done with other methodologies.  

    So, to get back to the original point, how do you translate story points to dollars?  I assume that if you are working with these venture capitalists that they would want to see that.

  • Jeff Sutherland

    This is not the best environment to clarify that Agile estimation is done completely differently and it cuts total project bidding and actual program management by 80% for fixed price projects for a good Scrum implementation in a CMMI Level 5 company. You will need to talk to their process leader to drill down into this.

    When you know points of a project and velocity of teams you divide points by velocity and you have team months. You then bid the appropriate cost of those team months. This is not only incredibly simple, but companies that do this well on fixed bid projects rapidly put their competition out of business.Unfortunately, most agile implementations suck so they are terrible examples. Over 50% of Scrum teams can’t get shippable code at the end of a sprint. No estimation strategy will help them with this level of engineering incompetence.That said, even a bad implementation of Scrum with give you a 35% reduction in project cost. We have data on hundreds of teams to show this. Even using hours you can get this but why would an investor put money into a 35% improvement. Investors want 10x and that was what Scrum was designed for. Why don’t project managers want 10x?

  • MarkS

    Just like Jeff wrote… this is incredibly simple. What’s more, I believe that at the _first_ estimation meeting you HAVE TO say “ok, so we have a story here that takes this ammount of work (not time!) and we assume that it is one story point”. We do not talk about time here. And the next story has to be estimated in comparison to the first one (“this story will take twice as much work so we estimate it as 2 story points”). Now we can do a first sprint with too much stories put into. After a sprint we can count how many points were _completely_ burned down and this is the velocity. From this moment we use (primary school level) calculations on how much some ammount of work costs us and maybe even how much the whole project will cost (if you do estimation meetings regulairly and spend some time to estimate future stories). This is my point of view and I stick to it. Thanks.

  • Big Show

    I like the article. However, my only thing is that he didn’t mention anything about “capacity”. What does it mean when one iteration there are 20 story points done and next 30? How do I know if my team is capable of 15 or 35 story points? Velocity han???

  • Juergen Heymann

    Jeff’s article states the well known fact that there is a huge variation in effort needed to complete a task depending on who is doing it. This factor remains whether you do story points or whatever other estimation technique. In fact it seems to me that story points cloud this issue as it suggests that the estimate is something pseudo-objective.
    What are your comments on this?

  • Prateek Shrivastava

    Question is how does a team improve?
    With velocity its simple and easy to measure. As velocity improves, this effectively means that team is getting more efficient in getting the work done. This however comes with condition that story point estimation should be consistent throughout.

    With hrs, team just continues to estimate work in hrs and we never know is team is improving or not.

  • Prateek Shrivastava

    On the other hand: this article did not meet my expectations..
    Sentences like “The best developer on a project takes one hour to complete a task while the worst developer takes 10 hours..” well, having a good developer is like having 90% of your problem solved. But that’s not usually the case. Not all companies are Facebook or google and able to attract the best talent in the industry. So, do you mean that we will not succeed, Mr Sutherland?

    Others like “story point estimation is better than hourly estimates as it is more accurate and has less variation”. How did you arrive at this conclusion?
    Data is only to back up a theory. In this article, I only saw data but not the theory (or wasn’t explained cleanly).

    • Klaus Bucka-Lassen

      I find the statement “Not all companies are Facebook or google and able to attract the best talent in the industry” interesting. Is it the name that attracts good developers? I wouldn’t think so. I don’t even think it’s the salary, but most of all it’s the possibility to work in an open, transparent, unconstrained environment where people can unfold and bring out their best. It is extremely fulfilling to do something you feel is good for the company you work for. Working in a surrounding that is managed through command & control on the contrary is not. Now, every company can provide such an environment, no matter whether it is an insurance company, a car manufacturer, a pharmacytical, or even a bank. You don’t have to be called Google or Facebook or Atlassian or Pixar or whatever. I can’t think of any industry where command and control is beneficial.

      In short: If you as a company honestly can say “We are Agile”, you will generally speaking be able to attract pretty good developers.

      Scrum is promising to help you reach that goal, moving from a command & control organization to an environment of facilitation.

      Unfortunately the transitions still fails relatively often because people tend to think “Scrum, that’s easy”. But it isn’t. It requires a mindset change and that’s one of the hardest things you can imagine – ask anybody that has tried to stop smoking or losing weight – two really easy concepts (just stop/take in less calories than go out), but implementation is really tough.

  • Confused ScrumMaster

    The biggest issue we seem to have is that the Product Owner gave the features team a chart showing story points and equivalent number of estimated hours to complete!  Now the PBI is littered with story points but the Sprint backlog contains the original estimate, hours remaining and hours completed.  Our reporting services are working at the moment so I am doing them manually.  We are running at a velocity of 65 story points with a 4 person team.  The number of tasks completed however goes up and down.  Percentage complete of sprint has never been higher than 65%.

  • ScrumIsJustAnotherBusiness

    Seriously?  The logic, glorifying storypoints over hours, is akin to saying that pricing a car in terms of no. of gold bars is better than pricing in terms dollars. And, how do you decide how many gold bars it is? Yes, you got it, by knowing how many dollars each gold bar costs. 

    Add to that, in real world, a team never sticks together (people change jobs, move teams, move to different projects, you don’t need all the people in that team for the next project… so on and so forth).  So measuring velocity of a particular team is useless unless you can guarantee that the same team (in all of its capacity) will be taking up your next project. Why, the velocity is combined velocity of all members and not invidual member velocity.

  • Learning

    There are some good points being made here. I am still quite new at this and I seem to struggle implementing it on a day to day basis especially in trying to separate what is practical from what is popular.

  • Jeff Sutherland

    Please review the original research that shows that humans are terrible at estimating hours. This was done by the Rand Corporation in the 1940’s for the Department of Defense. You can start by Googling the Delphi technique on Wikipedia. You might also review the data on project failure using this technique which is over 80% for any project over $3M. Do your own research.

  • Bkuehlhorn

    I was hopeful and sadly disappointed. Story points are a good size metric, but only if evaluating story points after the project is over. Dr. Jeff Sutherland suggests story point estimates are accurate. How does he know? There’s no indication estimates were validated by actuals. How does he evaluate actual story points?

    CMMI Level 5 companies should have Size and Effort parameters. Story Points make a good Size parameter when Estimates and Actual are matched. Effort needs to be collected, too. I would like to see their data and methods to determine actual story points.

  • Jeff Sutherland

    Systematic, the only Scrum CMMI Level 5 company, uses story points for estimation and has cut project management costs by 80% using agile planning practices. That is the major reason they offer fix price contracts at half the price of waterfall contracts. Please do your own research and look at the Scrum and CMMI papers in the IEEE Digital Library. You can also download them at http://scrum.jeffsutherland.com by clicking on “Jeff Sutherland’s Papers”.

  • Uberto Barbini

    We tried Story Points by-the-book but it didn’t work very well. Because of different people productivity, new requirements discovered on the way and general background noise, we weren’t able to estrapolate a “objective” velocity.
    On the other side, doing rough estimation of delivery dates is working very well for us. We constantely deliver in about ±10% of the estimate time. Interestly we deliver 10% early on the date as much as frequently as 10% late.

    “The management metric for project delivery needs to be a unit of production. ” -> I didn’t get the meaning of this UoP. Is the team productivity? or is it something from the management side?

  • http://scrum.jeffsutherland.com Jeff Sutherland

    At Systematic they deliver within 10% of estimates. It is a requirement for CMMI Level 5. However, if they use waterfall it costs twice as much and they have 40% more defects plus or minus 3 percent. If you measure hours you measure cost and you will get more cost. If you measure story points you measure features delivered and you get more features.

    • Uberto Barbini

      Thanks for the answer.
      That is an advantage for sure. :)

  • Luis Espinal

    And still, the article does not define what a story point is.