Revisiting Normalized Story Points

It has been just over 18 months since I published my most popular blog post to-date: "Normalized Story Points". As of this writing, it has received over 5,400 page views, thanks primarily to the reference it has from Dean Leffingwell's blog post from ScaledAgileFramework.com entitled "More on Normalizing Story Point Estimating".

As an Agile Coach who takes continuous improvement seriously, I have learned much and grown much over the last year and a half. Here are a few of the things I've learned regarding Normalized Story Points.

Know Why You Want Them

I've become a HUGE fan of David Hussman's "Dude's Law". Before you do anything, you need to know WHY you're doing it, not just HOW to do it.

The purpose of Normalized Story Pointing is to enable high-level estimation of work for a SAFe Release Train comprised of closely aligned teams, any one of which may pull a Feature off the Program Backlog for implementation. Normalized Story Points should not be used for:

1. Comparing teams against each other
2. Forcing teams to use the same story point size (so that "1 Story Point" means exactly the same thing to everyone forever and ever amen)
3. Defining "Story Point" to mean "Ideal Man-Day"
4. Any punitive purpose

If you don't know why you're using Normalized Story Points, don't use them. If your intended use of Normalized Story Points violates the mindset established by the Agile Manifesto, don't use them (although, if this is the case then you probably won't realize it unless someone points it out to you).

Beware the Ideal Man-Day

The 4-step process for getting started with Normalized Story Points involves using time as a reference. This works well because you're using a very small story, one with very little complexity, which means the primary factor for determining "bigness" is lapsed time. That being said, I've found that teams that start with this approach have a very, very difficult time recovering from the mindset of "1 point == 1 ideal man-day". This is bad for reasons.

This also leads to teams always using days of availability to forecast velocity rather than taking a rolling average velocity or "yesterday's weather" as an indicator of what they're likely to deliver. Data trumps theory; the approach to get started is based on theory, while average velocity is data.

Individuals and Interactions over Processes and Tools

Processes and tools exist to enhance and enable interactions between individuals, not replace them. As I've explained in a previous post, the process of Agile estimation is beneficial primarily because of the conversation that it fosters. It's a well-formed game that drives a shared mental model of the work for the entire team.

I've found that teams that use Normalized Story Points may skimp on the conversation, especially if one team ends up with work that another team has already estimated (after all, we all share the same definition of story point, right?). This was never the intent behind Normalized Story Points, but that doesn't mean it doesn't happen.

Beware the Recipe Trap

David Hussman explained that he uses "Dude's Law" to help teams avoid what he calls the "Recipe Trap" - doing something because that's what the recipe says to do, without understanding what the intended benefits are. When it comes to Normalized Story Points, the recipe trap is very real and very dangerous. If you use it, be very careful about how you explain and implement it with your teams. If your teams can't all pull from a common backlog; if you're going to use them to run punitive metrics; if you want to streamline your process by reducing meaningful conversation; or if you're looking to use it because that's what SAFe says to do, I plead with you to reconsider.

TL;DR: Normalized Story Points can be a useful tool for your organization, but it's not a one-size-fits-all solution. 

Understanding Agile Planning part 2: Decomposition Guidelines

I recently wrote a post explaining the process of Agile Planning. When done correctly, your work is decomposed to just the right level, just in time. If you haven't read this yet, please take a few minutes to familiarize yourself. I'll wait.

Good. Now that you understand the context, I want to address a concern that's been brought up recently: how do I know when to decompose my work? The short answer is always, but that's a little too ambiguous for most people. I'm going to share my guidelines here, and I'd ask that you share your advice in the comments.

Five Thresholds of Decomposition

Of course, as we decompose our work from high-level strategy (i.e. themes) to the low level work that team members execute on (i.e. stories or tasks), we rarely find that it fits neatly into 5 iterations of decomposition. Interestingly enough, that doesn't really matter. What does matter is the threshold that you establish for each level.

I'll start with User Stories, as that's the size that most people are already familiar with. Most teams have an upper-limit for how many Story Points a User Story can be in order for them to commit to it. Most of the time this is 8 points, though I've seen as high as 13 and as low as 3. Regardless, it's easy to see how the team has established a threshold for their User Stories. Whether you classify work larger than the team's threshold as a different type of work (Feature, Epic, etc.) or just call it a big User Story, the fact remains that it does not fit within the threshold for that level of planning. It must be decomposed.

Let's say that anything larger than 13 points is a Feature - but to what threshold? I would recommend something that can be done within 3 Sprints or less. Features are the lowest level of value as seen from the end-user's perspective, and end-to-end value should be production-ready with no more than 3 Sprints of development, depending on the product.

If your team's average velocity is 42, then your upper threshold for Features would be 126. That number's a little too precise for me, so let's call it 100. As Features become high priority, they should be broken into User Stories and prioritized for the coming Sprints' Planning ceremonies. Until that time, however, there is no need to decompose lower than the 100 point threshold.

Let's call Epics anything that take a quarter or less to deliver. This would be 13 weeks, so let's be conservative and call that six Sprints. We're now looking at 252 points, which can be rounded to 200, 250, or 400, depending on the scale you prefer. We now know that the work we're considering doing within the next quarter or two should be broken down into Epics that are within that threshold. While Epics may logically be broken down to under 100 points, we shouldn't feel compelled to split Epics that are already under our Epic threshold until they're a month or two out.

Anything larger than an Epic could be considered a Theme, with no upper-limit. You may decide that even Themes need an upper limit, but I generally don't include that in my recommendations.

Guidelines

This blog is more what you'd call guidelines than actual rules.

As I stated previously, these are simply guidelines. You may take a totally different approach. I find that some areas may define a Feature as something that takes two teams 4 weeks to do, knocking the threshold up to over 300. Others may decide that 100 is their Epic threshold. However you decide to define them, using thresholds as part of your working agreements make it very easy to tell when work needs to be decomposed and when it's better to wait so as to avoid rework or unnecessary work.

Introspective: Conversation Framework

We're all just faking it and hoping everything works out.

What's in a Number?
For at least half a year now, there has been a growing movement in the Agile Community for #NoEstimates - the concept that estimating the size of your work, regardless of method, is detrimental. The idea is intriguing and thought-provoking, which automatically gives it some degree of merit among those who are constantly striving to push the boundaries of Agile. The thoughts on the subject that most resonate with me on this subject come from Dan Mezick of Open Agile Adoption fame and Valerie Santillo, a blogger I met last year in D.C. and have kept in touch with since. What both of these thought leaders have said is that the conversation that leads to the estimate consensus is the most valuable aspect of the estimation process, and that great care should be taken to maintain that learning process.

The question then becomes: can you have a learning conversation and reach some sort of consensus as a team without an estimate to trigger that the consensus has been reached? My intent here is not to answer the question, but to illustrate how powerful this concept - using a conversation geared towards achieving consensus as a framework for learning - can be in practice.

Meetings, or Games?
Dan Mezick describes this consensus-driving conversation framework in the context of gamification. He posits that all meetings are games, and uses Jane McGonigal's definition of good games to show that meetings are ineffective because they are broken! A good game contains: a clear goal; clear rules; a way to get feedback; and opt-in participation. In a later post, Dan details How Games Deliver Happiness & Learning by putting gamification in terms of senses: a sense of control; a sense of progress; a sense of membership and community; and a sense of higher purpose and meaning.

Estimation as a Game
I propose that Story Point Estimation, particularly the Planning Poker approach or similar, meets the criteria for a good game as stated above. First, there is a clear goal: to agree on the "bigness" of each User Story. This is where my concern with the #NoEstimates movement comes into play, though I'm definitely open to their alternatives. Second, there are clear rules; the team has a Definition of Done for each Sprint, and each User Story has Acceptance Criteria to help the team understand what the User Story is (and isn't). Third, there is a way to get feedback through both conversation and revealing of Story Points. Conversation, in theory, would be good enough. However, experience has taught us that assumptions not stated in the initial conversation are revealed when discrepancies emerge as Points are shared. By having disparity in estimates, we are providing feedback that our assumptions differ. Fourth, the participation is opt-in - or, at least, it should be. Estimation should be done at a time and for a duration that the team agrees to. Nobody on the team should feel like they are being forced to participate. If there are, the Scrum Master or Agile Coach needs to step in, mine for conflict, and find out why the team member is disengaged so that the team can fix it.

Now let's look at it in terms of senses. First, there's the sense of control. The process of Planning Poker should, again in theory, prevent the process of estimation from being dominated by a loud minority. Every participant should feel some level of control over the direction of the conversation. Second, the sense of progress. As votes are cast and the numbers converge, we feel like we are nearing our goal for each Story. On a deeper level, we're growing closer as a team and gaining a more in-depth understanding of the system being built, which provides us a feeling of accomplishment and satisfaction. Third, the sense of membership and community, which should have been fostered prior to the activity and is reinforced throughout the conversation. Finally, the sense of higher purpose and meaning. This sense is driven by the nature of the work being discussed and estimated; a team that isn't opting-in may not feel this sense.

Every Meeting as a Game
What I've just walked you through are the conclusions I've started to form about how ALL conversations should be structured. It's the context with which I constructed the training material for the senior management that my fellow Coach and I were tasked with training on Agile. We started with the goal: how will each person change their behavior as a result of the paradigm shift we take them through? The mechanism we chose was the 5-Spoke Wheel Retrospective that we would end the training with. Starting with the end in mind, we created a framework for having the participants discuss the Principles of Agile and the Themes of Product Development Flow that would result in them debating with each other and examining the work they do on a daily, weekly, monthly, quarterly, and annual basis. As facilitators, we spoke for about a quarter of the time, opting for audience participation and group teaching instead.

I was shocked and amazed with not only how smoothly the class ran with so little structure, but with the amount of positive feedback we got afterwards! I then thought back to my early days with my first Scrum teams, how they would talk about how little time they spend in meetings now that they've gone Agile. When questioned about their Daily Scrum, Story Time (Backlog Grooming), Sprint Review & Demo, Sprint Retrospective, and Sprint Planning meetings, they would respond, "Oh, we don't really think of those as meetings; they're more active group conversations."

Framework for Success
Personally, I like Story Point Estimation and will continue to use it as a tool to help people bridge the gap from traditional (Waterfall) project structure and more Agile approaches. If they decide to do away with estimation altogether and can do it in such a way that the valuable conversation is not lost, I say go for it. I will also begin to encourage everyone to take a gamification approach to meetings with the goal that people not only look forward to them, they are insulted at the insinuation that their conversations are called "meetings". After all, meetings are dull, pointless, no-fun snooze-fests!

Normalized Story Points

UPDATE: 18 months later, I've reflected back on this topic in my post "Revisiting Normalized Story Points".

I was recently involved in a discussion on LinkedIn on the topic of Normalized Story Points and whether or not it was a good idea. What I noticed was that, despite what I think is a very clear explanation by the Scaled Agile Framework (SAFe), many people misunderstand both the process for normalizing Story Points and the intended benefit to be gleaned from this practice. If you have not yet, I encourage you to read the explanation in the “Iterations” abstract on the Scaled Agile Framework before reading on (the section entitled “Relative Estimating, Velocity and Normalizing Story Point Estimating”). That will provide the proper context for my thoughts, as a Certified SAFe Program Consultant and active Release Train Engineer, on what this means (and what it doesn’t).

First, how do we normalize Story Points? The gist of the process is this:

“1. For every developer tester on the team, give the team eight points (adjust for part timers)
2. Subtract one point for every team member vacation day and holiday
3. Find a small story that would take a about a half-day to code and a half-day to test and validate. Call it a 1.
4. Estimate every other story relative to that one.”

That’s basically it. The primary problem that teams run into is that they ignore step #4, along with the advice that follows these steps: “There is no need to recalibrate team estimating or velocities after that point.” This is a tool that has very specific benefits; it is intended for one-time use that allows for long-term benefit.

This means that teams within a SAFe organization are using a common practice to determine a baseline from which to begin relative estimation and, as such, will have Story Points that are roughly the same size. This will enable “just good enough” Story Point estimation of cross-team Features and Epics, which allows for more precise road-mapping and budgeting than we would otherwise have.

This does not mean that teams are estimating using Ideal Man-Days. They should still do relative estimation using the modified Fibonacci sequence. This also does not mean that roadmaps will be 100% precise; they will simply be more precise than what’s been available before. Budgeting will be precise in the sense that we float scope, not cost; however, the Epic Owner should only expect the Minimum Viable Product (MVP) and not necessarily every requested feature when the Epic has been initially estimated and not yet broken-down.

This means that teams establish how “big” a Story Point is early and don’t have to spend as much time figuring that out as a team. When I first became a Scrum Master, my team ended up having to re-baseline several times as they learned how to better write and slice User Stories. The alternative would have been either User Stories that were smaller than one point or multiple one-point Stories that were considerably variable in the amount of effort they actually required.

This does not mean that all teams will have exactly the same Story Point “size” or estimate exactly the same way. Every team is still unique and has their own dynamic that works for them. You will find, however, that the “size” of each team’s Story Point is close enough to allow for Program and Portfolio level estimating and planning.

This is not intended to take the place of established Agile practices for planning, splitting, and estimating the work that teams deliver. It is intended to get teams started in such a way that planning, splitting, and estimating can be done more simply and effectively across multiple teams. Once you establish your baseline, throw Ideal Man-Days out the window – it then becomes only a part of the criteria for relative estimation, along with complexity and uncertainty. Once you establish your initial velocity, throw the 8 Story Points per person per Sprint out the window – instead, use “yesterday’s weather”, current circumstances, and team growth to determine the number of points the team will commit to each Sprint.

Just like any other tool, metric, practice, meeting, or role that we use in Agile, the practices used for Normalizing Story Points have a stated benefit. We do not do things for the sake of doing them. Normalizing Story Points can have a tremendous positive impact on an organization practicing Agile at scale; a misunderstanding of Normalized Story Points can have a significant negative impact on that same organization.