Tell Me Why, not How

If you're familiar with the concept of User Stories then you've likely been exposed to the concept of a User Story "script". One of the most common scripts goes something like:

As a <who> I want <what> So that <why>

Looks simple enough, right? The problem is that most people who grew up in a non-Agile environment are not used to articulating the "why" in their requirements. They are used to capturing what it is that needs to built and how. Consequently, that's what usually end up making it into our stories, causing them to look like:

As a <who> I want <how> So that <what>

What's the difference? Let's look at an example.

Old Habits Die Hard

It's not uncommon to see a story that reads like, "As an administrator I want unique group IDs for tickets so that I can easily group tickets." On the surface it looks like a good user story. What's the "what"? The author would argue that it's "unique group IDs for tickets", but that's not the need. The need is to "easily group tickets". The author is trying to dictate how to meet that need in the user story. Those familiar with the INVEST model for user stories will recognize that this violates the "N" - Negotiable.

So we need to move the "what" into its place and establish a proper "why", which means we need to have a conversation. What valuable outcome is enabled by being able to group tickets together? Do you want static or dynamic grouping? Are the groups established by pre-established rules, or is the intent for users to group tickets however they like? Once we have our answers, we may end up with a better user story, something like, "As an administrator I want to easily group tickets so that I can sort and filter to easily find tickets of interest for a given task".

Why? Why?! WHY?!?

What have we gained by phrasing the story this way? A couple of things (at least). First, we have context for the story that will enable the team of professional problem-solvers to create the best solution for the problem within the bounds of the NFRs (Non-Functional Requirements) and FSRA (Future State Reference Architecture).

The second benefit is that we can better decompose the story. With the original wording, we would likely decompose the work into the layers of the system - creating a grouping cross-reference table structure first, then the business logic and virtual object code to associate tickets to groups, and finally the user interface layer for the administrator to actually use this functionality.

With the revised wording, we can leverage the advice from Agile Learning Labs at SmallerStories.com and split our story into smaller stories that are vertical slices through the entire system. This enables us to deliver value sooner and test our architecture as it's being built instead of all at the end.

Living in Reality

This is the most common anti-pattern that I've seen in the real world of user stories. What anti-patterns have you seen? What advice do you have for overcoming user story anti-patterns? Is this even really an anti-pattern?

"I Like that Old Time Rock 'n Roll"

I'm am constantly amazed with the power of first impressions. As much as we may try to avoid it, the initial impressions we get when we're first introduced to a person, place, or idea tend to stick with us for the rest of our lives. This sentiment is embodied in the classic Bob Segal hit, "Old Time Rock and Roll". The song starts with its thesis:

"Just take those old records off the shelf, I'll sit and listen to 'em by myself. Today's music ain't got the same soul; I like that old time Rock 'n Roll"

This same phenomenon happens in Agile as well. When people think of Agile Transformations being hindered by first impressions, they usually jump to those who "didn't do it right", who failed and were left with a bitter taste in their mouth. I would argue that, in the long-term, the more hazardous first impression comes from those who have greater-than-anticipated success with the first thing they try. From that point on Agile is defined solely by what they did the first time.

Let's say your first approach was Scrum - and I mean by-the-Scrum-Guide Scrum. It works so well for you that you scoff at Kanban, DevOps, or anything else that violates your 5 ceremony, 3 role, pure and holy Agile approach. Nevermind that you are supposed to be looking for ways to improve every Sprint. Nevermind that the best way to improve team performance is adjustments to the process, not the people. "I'm sorry, you don't use time boxes?" "How do you plan without using User Stories?" "Clearly you don't know what Agile means."

I find it telling that the first line in the Agile Manifesto states that "we are uncovering better ways of developing software by doing it and helping others do it" (AgileManifesto.org, emphasis added). They didn't say that they had uncovered the best way of developing software, they were in the never-ending process of uncovering better ways. They didn't come up with the Manifesto by getting doctorate degrees and jumping into a think tank, they came up with it based on their experiences in doing it and helping others.

I find it highly narcissistic for anyone to have claimed the best way to do anything. Given how long mankind has been on the earth and the fact that nothing has yet been proven beyond improvement, saying that you've "figured it out" means you're either selling something, deluding yourself, or both. Those who really get Agile know that it's about the journey, not the destination. If you keep reliving the glory days of your first Scrum team then you'll never give proper attention to challenging the status quo and innovating the next big idea for delivering software in a better way.

I know, I know, it's hard to give up those first impressions, the glory days, the golden years when everything was perfect and your selective memory hadn't kicked in yet. But you have to if you want long-term success. It's risky business, but somebody's got to do it.

Understanding Agile Planning part 2: Decomposition Guidelines

I recently wrote a post explaining the process of Agile Planning. When done correctly, your work is decomposed to just the right level, just in time. If you haven't read this yet, please take a few minutes to familiarize yourself. I'll wait.

Good. Now that you understand the context, I want to address a concern that's been brought up recently: how do I know when to decompose my work? The short answer is always, but that's a little too ambiguous for most people. I'm going to share my guidelines here, and I'd ask that you share your advice in the comments.

Five Thresholds of Decomposition

Of course, as we decompose our work from high-level strategy (i.e. themes) to the low level work that team members execute on (i.e. stories or tasks), we rarely find that it fits neatly into 5 iterations of decomposition. Interestingly enough, that doesn't really matter. What does matter is the threshold that you establish for each level.

I'll start with User Stories, as that's the size that most people are already familiar with. Most teams have an upper-limit for how many Story Points a User Story can be in order for them to commit to it. Most of the time this is 8 points, though I've seen as high as 13 and as low as 3. Regardless, it's easy to see how the team has established a threshold for their User Stories. Whether you classify work larger than the team's threshold as a different type of work (Feature, Epic, etc.) or just call it a big User Story, the fact remains that it does not fit within the threshold for that level of planning. It must be decomposed.

Let's say that anything larger than 13 points is a Feature - but to what threshold? I would recommend something that can be done within 3 Sprints or less. Features are the lowest level of value as seen from the end-user's perspective, and end-to-end value should be production-ready with no more than 3 Sprints of development, depending on the product.

If your team's average velocity is 42, then your upper threshold for Features would be 126. That number's a little too precise for me, so let's call it 100. As Features become high priority, they should be broken into User Stories and prioritized for the coming Sprints' Planning ceremonies. Until that time, however, there is no need to decompose lower than the 100 point threshold.

Let's call Epics anything that take a quarter or less to deliver. This would be 13 weeks, so let's be conservative and call that six Sprints. We're now looking at 252 points, which can be rounded to 200, 250, or 400, depending on the scale you prefer. We now know that the work we're considering doing within the next quarter or two should be broken down into Epics that are within that threshold. While Epics may logically be broken down to under 100 points, we shouldn't feel compelled to split Epics that are already under our Epic threshold until they're a month or two out.

Anything larger than an Epic could be considered a Theme, with no upper-limit. You may decide that even Themes need an upper limit, but I generally don't include that in my recommendations.

Guidelines

This blog is more what you'd call guidelines than actual rules.

As I stated previously, these are simply guidelines. You may take a totally different approach. I find that some areas may define a Feature as something that takes two teams 4 weeks to do, knocking the threshold up to over 300. Others may decide that 100 is their Epic threshold. However you decide to define them, using thresholds as part of your working agreements make it very easy to tell when work needs to be decomposed and when it's better to wait so as to avoid rework or unnecessary work.

Understanding Agile Planning

One common myth about Agile is that those who practice Agile don't plan. I've found the opposite to be true: those who truly understand Agile are planning all the time. The difference is that the planning they do is much more lightweight. They also anticipate change, rather than pretend it doesn't exist, resulting in plans that are living documents instead of notarized relics. Their plans are always accurate, but with varying levels of precision. How do they do this? Using one of the simplest, most powerful planning tools out there: the backlog.

The Backlog
A backlog is really nothing more than a prioritized list. If you've ever made a To-Do List (or its popular cousin, the Honey-Do List) then you've made a backlog. If your backlog was short, you probably got a lot done and felt a great swell of accomplishment. If your backlog was exhaustive and poorly prioritized, you probably got overwhelmed and did little to "move the needle".

Levels of Planning
I'm tackling this from an enterprise view of Scrum, but the concepts are the same regardless of how many levels you use or the terminology you pick. In general, an enterprise using Scrum should have five levels of planning.

Levels of Planning

I'm not the first to notice these five levels, so I can't take credit for the concept. The idea is that you start with a Vision that serves as your "True North". Everything that you do should align with your Vision, and your Vision statement should be updated infrequently. At this level, you're looking at Themes or Strategies that execute the Vision at a very high level.

Everything in your Roadmap - lets call them Epics - should align to one or more Theme. You may call it something different, and the word Epic may mean something different to you. When I say Epic, I refer to a set of one or more Features that provide significant strategical value to your organization. Waterfall project charters usually consist of one or more Epics as I've defined them here. Your Roadmap may go out 3-5 years or more, but becomes more "squishy" the further out you go. Roadmaps should be revisited at least quarterly and updated whenever necessary.

Releases should align to the Roadmap and should consist of one or more Features. When I say "Release", I'm referring to a production release with significant functionality. If you're in a shop that employs Continuous Delivery, this would obviously not refer to the type of release that happens multiple times each day; perhaps the word "Release" doesn't even make sense in that context. Regardless of the term, this is the level that is easiest for end-users to understand. Everything above this level is too abstract, and everything below this level is too granular.

Sprints should have goals that align with the Release plan. The Scrum teams do this by committing to User Stories that contribute to the completion of the next Releases Feature(s). My recommendation for Sprint Duration is two weeks. I've tried 3-week, 4-week, and month-long Sprints and I didn't care for any of them. Two weeks seems to be the sweet spot. Changes to the Sprint plan should be kept to a minimum, especially if the team is a less-mature Agile team. Again, the mechanics of this change for Kanban, DevOps, or similar teams.

At the lowest level, the team aligns in their Daily Scrum on how they will work together to accomplish the Sprint's goals. They may be aligning in the context of tasks, which may or may not be planned out at the beginning of the Sprint, but they are not likely to be discussing their daily work in terms more abstract than User Stories. After all, they know how their Stories are tied to Features, how those Features are tied to Epics, and how those Epics are tied to Themes; there's no reason to revisit the higher levels every day.

The Planning Engine
The mechanism for moving work intake to something that's actionable is actually quite similar across all levels. The cadence, level-of-detail, and people involved may differ, but the process is essentially the same.

1. Discuss the item
2. Estimate the item
3. Prioritize the item
4. Implement or Decompose the item
• If Decomposing, the sub-items are fed to the next level down and the process repeats at step #1.

At the team level, this is done in the Backlog Grooming sessions (or, as I like to call them, "Story Time"). The same concept can and should be used at the other levels of Planning, simply modified to be appropriate for said level. For example, you may have more executive people meeting quarterly to groom the Epic backlog; you may have representatives from each Scrum team in a product area get together monthly to groom the Feature backlog; and you may meet as a Scrum team each Sprint to groom the Product (User Story) backlog. Use whatever cadence and team members that makes sense for your organization and circumstances.

A SAFe Approach

If you're familiar with the Scaled Agile Framework (SAFe), then you know that what I just described is accounted for in their Big Picture. If you wanted more advice on how to implement this planning engine then I recommend you view the abstracts found on their website. I've found their advice to be very useful and pragmatic.

You can implement the five levels of planning with a backlog-driven planning engine without SAFe. Indeed, there are many non-SAFe Agile companies that have established an Agile planning process before the advent of SAFe. I'm simply suggesting that, if you're new to this and don't know where to start or if you're in the middle of this and struggling with what you have, check out what SAFe has to offer. Take what works for you and toss the rest.

A Rose by Any Other Name...

As I stated at the beginning, mine is only one perspective on Agile Planning, and the terms I use may be different (or defined differently) than what you use in your organization. I would love to get your perspective on how Agile Planning can be done. After all, I'm always on the lookout for better, more Agile ways to do things!

Introspective: Conversation Framework

We're all just faking it and hoping everything works out.

What's in a Number?
For at least half a year now, there has been a growing movement in the Agile Community for #NoEstimates - the concept that estimating the size of your work, regardless of method, is detrimental. The idea is intriguing and thought-provoking, which automatically gives it some degree of merit among those who are constantly striving to push the boundaries of Agile. The thoughts on the subject that most resonate with me on this subject come from Dan Mezick of Open Agile Adoption fame and Valerie Santillo, a blogger I met last year in D.C. and have kept in touch with since. What both of these thought leaders have said is that the conversation that leads to the estimate consensus is the most valuable aspect of the estimation process, and that great care should be taken to maintain that learning process.

The question then becomes: can you have a learning conversation and reach some sort of consensus as a team without an estimate to trigger that the consensus has been reached? My intent here is not to answer the question, but to illustrate how powerful this concept - using a conversation geared towards achieving consensus as a framework for learning - can be in practice.

Meetings, or Games?
Dan Mezick describes this consensus-driving conversation framework in the context of gamification. He posits that all meetings are games, and uses Jane McGonigal's definition of good games to show that meetings are ineffective because they are broken! A good game contains: a clear goal; clear rules; a way to get feedback; and opt-in participation. In a later post, Dan details How Games Deliver Happiness & Learning by putting gamification in terms of senses: a sense of control; a sense of progress; a sense of membership and community; and a sense of higher purpose and meaning.

Estimation as a Game
I propose that Story Point Estimation, particularly the Planning Poker approach or similar, meets the criteria for a good game as stated above. First, there is a clear goal: to agree on the "bigness" of each User Story. This is where my concern with the #NoEstimates movement comes into play, though I'm definitely open to their alternatives. Second, there are clear rules; the team has a Definition of Done for each Sprint, and each User Story has Acceptance Criteria to help the team understand what the User Story is (and isn't). Third, there is a way to get feedback through both conversation and revealing of Story Points. Conversation, in theory, would be good enough. However, experience has taught us that assumptions not stated in the initial conversation are revealed when discrepancies emerge as Points are shared. By having disparity in estimates, we are providing feedback that our assumptions differ. Fourth, the participation is opt-in - or, at least, it should be. Estimation should be done at a time and for a duration that the team agrees to. Nobody on the team should feel like they are being forced to participate. If there are, the Scrum Master or Agile Coach needs to step in, mine for conflict, and find out why the team member is disengaged so that the team can fix it.

Now let's look at it in terms of senses. First, there's the sense of control. The process of Planning Poker should, again in theory, prevent the process of estimation from being dominated by a loud minority. Every participant should feel some level of control over the direction of the conversation. Second, the sense of progress. As votes are cast and the numbers converge, we feel like we are nearing our goal for each Story. On a deeper level, we're growing closer as a team and gaining a more in-depth understanding of the system being built, which provides us a feeling of accomplishment and satisfaction. Third, the sense of membership and community, which should have been fostered prior to the activity and is reinforced throughout the conversation. Finally, the sense of higher purpose and meaning. This sense is driven by the nature of the work being discussed and estimated; a team that isn't opting-in may not feel this sense.

Every Meeting as a Game
What I've just walked you through are the conclusions I've started to form about how ALL conversations should be structured. It's the context with which I constructed the training material for the senior management that my fellow Coach and I were tasked with training on Agile. We started with the goal: how will each person change their behavior as a result of the paradigm shift we take them through? The mechanism we chose was the 5-Spoke Wheel Retrospective that we would end the training with. Starting with the end in mind, we created a framework for having the participants discuss the Principles of Agile and the Themes of Product Development Flow that would result in them debating with each other and examining the work they do on a daily, weekly, monthly, quarterly, and annual basis. As facilitators, we spoke for about a quarter of the time, opting for audience participation and group teaching instead.

I was shocked and amazed with not only how smoothly the class ran with so little structure, but with the amount of positive feedback we got afterwards! I then thought back to my early days with my first Scrum teams, how they would talk about how little time they spend in meetings now that they've gone Agile. When questioned about their Daily Scrum, Story Time (Backlog Grooming), Sprint Review & Demo, Sprint Retrospective, and Sprint Planning meetings, they would respond, "Oh, we don't really think of those as meetings; they're more active group conversations."

Framework for Success
Personally, I like Story Point Estimation and will continue to use it as a tool to help people bridge the gap from traditional (Waterfall) project structure and more Agile approaches. If they decide to do away with estimation altogether and can do it in such a way that the valuable conversation is not lost, I say go for it. I will also begin to encourage everyone to take a gamification approach to meetings with the goal that people not only look forward to them, they are insulted at the insinuation that their conversations are called "meetings". After all, meetings are dull, pointless, no-fun snooze-fests!

Normalized Story Points

UPDATE: 18 months later, I've reflected back on this topic in my post "Revisiting Normalized Story Points".

I was recently involved in a discussion on LinkedIn on the topic of Normalized Story Points and whether or not it was a good idea. What I noticed was that, despite what I think is a very clear explanation by the Scaled Agile Framework (SAFe), many people misunderstand both the process for normalizing Story Points and the intended benefit to be gleaned from this practice. If you have not yet, I encourage you to read the explanation in the “Iterations” abstract on the Scaled Agile Framework before reading on (the section entitled “Relative Estimating, Velocity and Normalizing Story Point Estimating”). That will provide the proper context for my thoughts, as a Certified SAFe Program Consultant and active Release Train Engineer, on what this means (and what it doesn’t).

First, how do we normalize Story Points? The gist of the process is this:

“1. For every developer tester on the team, give the team eight points (adjust for part timers)
2. Subtract one point for every team member vacation day and holiday
3. Find a small story that would take a about a half-day to code and a half-day to test and validate. Call it a 1.
4. Estimate every other story relative to that one.”

That’s basically it. The primary problem that teams run into is that they ignore step #4, along with the advice that follows these steps: “There is no need to recalibrate team estimating or velocities after that point.” This is a tool that has very specific benefits; it is intended for one-time use that allows for long-term benefit.

This means that teams within a SAFe organization are using a common practice to determine a baseline from which to begin relative estimation and, as such, will have Story Points that are roughly the same size. This will enable “just good enough” Story Point estimation of cross-team Features and Epics, which allows for more precise road-mapping and budgeting than we would otherwise have.

This does not mean that teams are estimating using Ideal Man-Days. They should still do relative estimation using the modified Fibonacci sequence. This also does not mean that roadmaps will be 100% precise; they will simply be more precise than what’s been available before. Budgeting will be precise in the sense that we float scope, not cost; however, the Epic Owner should only expect the Minimum Viable Product (MVP) and not necessarily every requested feature when the Epic has been initially estimated and not yet broken-down.

This means that teams establish how “big” a Story Point is early and don’t have to spend as much time figuring that out as a team. When I first became a Scrum Master, my team ended up having to re-baseline several times as they learned how to better write and slice User Stories. The alternative would have been either User Stories that were smaller than one point or multiple one-point Stories that were considerably variable in the amount of effort they actually required.

This does not mean that all teams will have exactly the same Story Point “size” or estimate exactly the same way. Every team is still unique and has their own dynamic that works for them. You will find, however, that the “size” of each team’s Story Point is close enough to allow for Program and Portfolio level estimating and planning.

This is not intended to take the place of established Agile practices for planning, splitting, and estimating the work that teams deliver. It is intended to get teams started in such a way that planning, splitting, and estimating can be done more simply and effectively across multiple teams. Once you establish your baseline, throw Ideal Man-Days out the window – it then becomes only a part of the criteria for relative estimation, along with complexity and uncertainty. Once you establish your initial velocity, throw the 8 Story Points per person per Sprint out the window – instead, use “yesterday’s weather”, current circumstances, and team growth to determine the number of points the team will commit to each Sprint.

Just like any other tool, metric, practice, meeting, or role that we use in Agile, the practices used for Normalizing Story Points have a stated benefit. We do not do things for the sake of doing them. Normalizing Story Points can have a tremendous positive impact on an organization practicing Agile at scale; a misunderstanding of Normalized Story Points can have a significant negative impact on that same organization.

Conversation

Many Scrum practitioners don't realize that Scrum did not invent the idea of User Stories - XP (eXtreme Programming) did. They are, however, often familiar with Ron Jeffries' "3 C's of a User Story", namely Card, Conversation, and Confirmation. The more I train teams on User Stories, the more convinced I am that the Conversation aspect is the most important of the three, and it is applicable to all things Agile, not just User Stories.

I recently facilitated PSI Planning for my area's Agile Release Train (Potentially Shippable Increment). I’ve noticed that, with each new PSI, the teams become more concerned with extensive pre-planning. They try to get more out of the meeting by doing as much of the actual planning before arriving at the meeting. Unfortunately, they do not understand that the primary reason for PSI Planning is not to generate a plan as much as it is to have conversations. Teams must communicate and align with each other and their customers. They must examine their work to determine what dependencies and risks are inherent in the work so that they can communicate, manage, and mitigate them as much as possible. This process happens to produce a better plan, which is why we do it. However, those who come to the event with pre-determined plans often find they must be reworked or scrapped altogether.

Rework manifests itself as delay, and delay indicates waste. Agilists hate waste. In fact, an Agilist’s views on waste and rework can be found in the 10^th principle that accompanies the Agile Manifesto: “Simplicity – the art of maximizing the amount of work not done – is essential.” How do conversations help prevent rework? If we talk before acting then we get a better sense of the truth, established in an environment of shared context. Conversations mean multiple perspectives, which bring difficult questions and challenge our assumptions. They force us to produce something as a group that is better than could ever be produced in a silo. They reinforce the paradigm of “just enough, just in time” instead of “big up-front”.

Once I realized that conversations are integral to Agile, I began to review the tools and practices within Agile, specifically Scrum, for conversations. All of the prescribed ceremonies – Daily Scrum, Sprint Review, Sprint Retrospective, and Sprint Planning – center on team conversations. Pair Programming and Test-Driven Development are XP practices that rely heavily on communication, both within the development team as well as between the developers, testers, and customers. A good Scrum Master helps the team most by facilitating conversations, and the Product Owner’s backlog is meaningless if they don’t ensure the team understands the work. There are conversations baked into literally every aspect of Scrum!

Conversations are often overlooked because people do not view simply having the conversation as an objective in most cases. They focus on the stated outputs: plans; acceptance criteria; status updates; confirmation of stories completed; etc. Consequently, it is not uncommon for teams to find alternative means to deliver these results without the “overhead” of the conversations. They rely heavily on tools and individual efforts to maximize their time without hindering the team with human interaction. They may, at first, feel an increase in productivity as the perceived burden is lifted. Yet, inevitably, they find themselves struggling to deliver quality quickly and do not understand how Agile could betray them so. Hopefully they have a good Scrum Master or Agile Coach watching them adapt in the wrong direction that can have them take a step back so they can see the error of their ways. If not, it becomes easy to blame their frustration on Agile and revert back to their comfort zone of more “traditional” methodologies.

As a safeguard, I recommend teams, and especially Scrum Masters, go back to their roots frequently to make sure they’re staying true to the founding principles that make Agile methodologies so great. Remember that “business people and developers must work together daily throughout the project,” and that “the most efficient and effective method of conveying information to and within a development team is face-to-face conversation.” Don’t just blindly follow a framework or practice; instead, try to understand why you are asked to do things a certain way. If the purpose that you perceive is not aligned with the Agile manifesto and its accompanying principles, you’re probably wrong. Acknowledge your failure, “Fail Fast”, and change your approach so that it fits the Agile paradigm. It may seem counter-intuitive at first, but it will pay off in the long run.