Caution: Spikes

                Several misconceptions exist around the concept of a spike in Agile Development.  Spikes are an essential part of the development cycle but are inevitably either over- or under-used due to a failure to understand why we create spikes.  Hopefully I can help shed some light on how to use spikes to best drive business value and when to eschew them in favor of directly driving business value through stories.

                Taken from the Scaled Agile Framework website (www.scaledagileframwork.com): “the purpose of a spike is to gain the knowledge necessary to reduce the risk of a technical approach, better understand a requirement, or increase the reliability of a story estimate”.  So that seems fairly straightforward, right?  Unfortunately, due to the nature of human language, different people draw very different conclusions from the same source, taking a fairly simple, direct statement and completely misrepresenting it in ways the author never dreamed of.

                Let’s take the last two parts of that statement first: “better understand a requirement” and “increase the reliability of a story estimate”.  This means that if a team comes across a story that has so many unknowns they can’t accurately provide an estimate for the story, then a spike is in order to perform some research to determine how to best tackle the story.  This research may mean meeting with the end users to get more in-depth knowledge of the business process involved, meeting with other developers to determine the best way to implement a solution, working through external dependencies with other teams or many other activities too numerous to list here.  Performing this research will provide a better understanding of the given acceptance criteria, thereby raising the comfort level of the scrum team.  So how is this one botched?  Developers get spike-happy.   It is not uncommon for developers to become uncomfortable even with small levels of uncertainty and create spikes unnecessarily clogging up the backlog.  As a general rule of thumb, if you are creating a spike for an hour or two worth of research, it is probably not necessary to create that spike.

                Climbing further down this particular rabbit-hole, one must ask the question, “Why does it matter how many spikes I create if it helps make the story more accurate?”  The answer to this question is alluded to in the title of this article.  Customers do not care about spikes.  A spike by its nature delivers no business value and is generally throwaway if anything is actually developed during it.  As a result, it is important to minimize the spikes created unless necessary. 

When faced with uncertainty, the team must assess the level of unknowns in the story and make a determination.  Create a spike?  Perhaps increase the story points a little to account for some possible research needed?  Locate an SME on the story in order to provide feedback on the complexity of the story while pointing it?  The last option is preferable as it eliminates the unknowns immediately and gives the team a reasonable comfort level while actually pointing the story.  In this instance, it is important to be proactive and make sure that SME is invited to the discussion of the story or consulted with beforehand.  The second option of buffering the story points is not the optimal solution but is still preferable to creating a spike since most likely both the spike and story will not be completed in the same sprint.  This delays business value delivery to the customer.  On the downside, it still increases the size of the story, potentially making it too large to be taken in the preferred sprint.

Moving on to the first part of the definition of a spike (in case you’ve forgotten already, “gain the knowledge necessary to reduce the risk of a technical approach”), a spike may be created if a new technology has been decided on and some research or a proof of concept (POC) may be needed in order to proceed forward with development on the stories in question.  I have observed a couple of key pitfalls when dealing with this particular kind of spike that should be avoided if the spike is to be effective in the long-term. 

The first pitfall when dealing with POCs is the failure to time-box it properly.  A POC is throwaway by nature.  Please do not spend more time on it than necessary before completing the story.  Specify a time-box beforehand when planning the spike and ensure that once that time-box is met, the story that initiated the spike is revisited (Many sources recommend specifying the time-box in days or even hours to ensure that the developers involved are actively keeping track of the spike’s time).  If at that point, too many unknowns still remain in order to point the story, then another spike may be needed.  It is critical that the spike is kept as short as possible since again, no business value is being delivered.  That time-box is important because developers typically like to develop (go figure…) and may want to get that POC completely working.  Guess what?  That POC is junk.  Throw it away once you’ve gotten what you need from it and work on something the business cares about.

This leads me to the second pitfall.  Don’t turn the POC into the story.  If the developer starts actually coding the story, then why are we doing a spike?  This is risky for a couple of reasons.  For one, typically POCs are very short-lived and ad-hoc, leading to coding them with looser standards than we might have for our flagship applications.  If the POC starts to turn into the actual application, there are likely some bad habits that do not need to be carried forth.  Another concern with mutating a POC into story is hidden in intrinsic idea that we are in agile development and as such, things change.  The developer runs the risk of working on an actual story, integrating into the environment, and then ultimately undoing it when priorities shift or the technology being used doesn’t pan out.  And I haven’t met a developer yet that loves rework.  If you are still trying to decide what color to paint your house, you are probably not going to just start painting the front of your house with the first color you find that you sort of like the look of, right?   The important thing to remember with POCs is to keep them simple and keep them focused on the key unknowns that are being solved for.  Your POC does not need to adhere to all coding standards.  It does not need to have a full suite of J-Units.  It does not need to look pretty.  Heck, it really doesn’t even have to completely work, as long as you’ve gotten what you need out of it. 

In summary, spikes are absolutely important when used properly but are easily abused.  A good rule of thumb is not to spend more than a day or two on a spike unless you are dealing with a large number of complex unknowns in which you may have a few spikes to deal with.  Use your best judgment and remember than your customer does not care about these spikes.  Treat them like the begrudging necessity that they are and not as your top tool in the tackle box.