Tuesday, November 19, 2013

When NOT to Use Agile

When HealthCare.gov first rolled out, everyone experienced how terribly it was built. Many Agile enthusiasts cried out, “If only they had used Agile…!” We have since learned that HealthCare.gov used Scrum. It’s uncertain how superficial their adoption was. Michael Daconta has already done a great job describing their poor use of Agile methods in the GCN blog, and NPR has published findings from the McKinsley & Co. consultants who were brought on in March to do a progress report.

My focus here is not about the degree to which Agile was or was not properly used for HealthCare.gov. My focus is to debate under what circumstances, if any, Agile will not work. When would you NOT use Agile?
My assumptions for this debate are as follows:
  1. Agile is any processes, practices, tools, and frameworks that comply with the Agile Manifesto and its 12 Principles.
  2. In the spirit of healthy debate, I assume the audience to be open to the possibility of new and different ways of doing things. This applies particularly to both Agile skeptics and to those entrenched in a specific Agile approach.
  3. The ideas in this discussion are not etched in stone. I encourage you to share your own ideas so we can learn and grow together.

Possible Agile Risks

I was recently invited to an introductory Agile presentation. The audience was Project Managers with varying Agile knowledge and experience. It facilitated a conversation for answering questions and resolving concerns around Agile adoption. About halfway through, there was a slide that had five “Possible Agile Risks”:
  1. Business or management commitment
    • Agile requires an increased level of commitment from the business and IT management to be successful
  2. Remote or distributed teams
    • Agile benefits from close collaboration on a consistent basis. Distributed teams pose a challenge in the effort to do this effectively
  3. Remote product owners
    • The business must be consistently involved, so a remote product owner may be challenging to engage
  4. Balance delivering new features with high-quality, sustainable code
    • Agile delivers new features frequently, but this should never come at the expense of creating low quality code
  5. Team not provided adequate guidance as they implement Agile
    • Anything new will take time to get good at, it is important to supply the team with training and guidance so they can improve quickly

This disclaimer followed these risks: “These risks do not preclude teams from applying Agile; they just need to be addressed in the scenarios they are present.”

Before we get to my reaction, let’s walk through these risks. Let’s identify why they’re risky in an Agile environment.

Business or Management Commitment

A deep understanding of the business is essential to the success of the software that supports it. In Agile methodologies, this means a dedicated person who has a deep understanding of the customer’s needs, often called a “Product Owner”. Traditionally, this person is seen as “from the business side”. However, getting someone from the “business side” isn’t always possible and is, arguably, unnecessary.

In my area we have Business Analysts who understand the business needs. They are bilingual in business and IT speech. Our developers and testers have a deep business understanding of the business because they’ve been supporting it for a long time. Some Agile practitioners feel the Product Owner role is unnecessary; they believe the team should understand the customers’ needs better than anyone. The team can, therefore, manage the work themselves.

In Agile, management is not a clearly defined position. Managers may feel their job is threatened by a “self-managing” team. A manager must have the team’s best interests at heart. They must become Lean-Thinking Servant Leaders, not Command-and-Control Managers. This is a cultural shift from Waterfall. Agile teams without Servant Leader management will plateau until they are allowed to “self manage”. However, that doesn’t mean they can’t find some measure of success.

Remote or Distributed Teams

This is one of the first and most frequent challenges brought up when introducing Agile. A team is distributed when its members are not capable of frequent face-to-face conversation. Lack of co-location for an Agile team is a challenge, no question. However, modern tools such as IM-enabled ad-hoc video conferencing, digital code review tools, etc. make long-distance coordination much simpler. It may be hard to work out timing with time-zone differences, but it’s possible to have a time overlap for some real-time collaboration. This may result in having a tech lead that takes the brunt of the development’s complexity to reduce the need for collaboration among the other team members, though this will limit the team’s overall productivity.

Remote Product Owners

For this I say, “See my previous 2 sections.” However, perhaps there are situations where the team is new to the product? Perhaps the SMEs are remote? Even worse, what if those remote SMEs do not dedicate time to work with the development team? Wait, what on earth are you thinking? Seriously, this is a bad idea, regardless of methodology. You are stacking the deck against yourself and setting the team up to fail.

Balance New Features with Quality Code

One of the 12 Agile Principles states, “Continuous attention to technical excellence and good design enhances agility.” Delivering new features at the expense of quality is the technical equivalent of being “penny wise, pound foolish”. This may be faster short-term, but the long-term consequences are a slow-down of delivery. Without addressing the quality of the application, it will eventually be cheaper to replace the application than to fix it. This isn’t unique to Agile. This brings me to my initial reaction to ‘Possible Agile Risks’.

My reaction to "Possible Agile Risks"

When I first saw this list, my reaction was that the first four are not “Possible Agile Risks”. Instead, they are “Possible Software Development Risks”! The four risks of commitment, distributed teams, remote product owners, and quality are often associated with Agile development because Agile brings risks to light early. Agile makes it much more painful to ignore risks than to address them. They are not less risky with non-Agile approaches. In fact, the realization of these risks is more likely in a non-Agile approach. In Waterfall projects, risks manifest so late in the process that they are painful and expensive to fix. The first four are risks to all projects; but not necessarily the fifth and final risk on the list.

Agile Training and Guidance

Without proper training and guidance, it is difficult for those unfamiliar with the Agile paradigm to be successful with an Agile approach. Most people new to Software Development find the Agile way of thinking to be a more intuitive and human-centered. In those cases, there is less risk because less training is needed.

Those entrenched with Waterfall approaches will find it difficult to switch gears. This is the only risk I could concede as being a “Potential Agile Risk”.

Where’s the Risk?

So there are my thoughts on five risks commonly thought to make Agile less successful. I have yet to find a risk that is made any less risky by Waterfall. Does that mean Agile’s guaranteed to succeed? Absolutely not. In fact, Agile will push you to your limits. Then, right as you’re starting to feel comfortable, it pushes you some more. However, Agile will consequently turn your team into the best team they can be. Agile will expose risks and inefficiencies sooner. It will also make them more painful. As long as you don’t ignore the pain, taking care of those risks and inefficiencies will be less expensive because they are dealt with early.

But hey, maybe I’m wrong. What risks have you heard associated with Agile? What situations are wholly inappropriate for the basic tenants of the Agile Manifesto?

Tuesday, November 12, 2013

Agile Complements

I was recently asked to explain the relationship between Agile and Innovation. I was intrigued by the question as I've never really tied the two together. In thinking about it, I've concluded that Agile complements many things, but it doesn't necessarily drive any of them.

The Role of Practices

The Agile Manifesto clearly states that one who considers themselves Agile values "Individuals and Interactions over Processes and Tools". However, we know that there is some value inherent in Processes and Tools, primarily in the behaviors that they reinforce. Given it's popularity, let's take Scrum as an example. Scrum has a highly prescriptive process that still manages to leave a lot up to the team. Sprint Planning reinforces commitments and transparency; the Daily Scrum reinforces face-to-face communication and self-managing teams; Sprint Reviews reinforce feedback loops within the team, as well as with stakeholders and customers; Sprint Retrospectives reinforce continuous improvement; the Sprints themselves reinforce small time-boxes; and on and on and on.

If you want to reinforce something positive at work, whether it be innovation, elimination of waste, any of the items listed below, or pretty much anything else, you can find an Agile process, tool, or practice that will reinforce it. With Innovation, you can take the practices of Capacity Allocation and Spikes to explicitly include time and effort for innovation into your process. This is assuming we're talking about developing innovative things, in addition to a Retrospective or other feedback mechanism for innovating the process and team themselves.

However, it is possible - and happens all too frequently - that teams adopt these processes, tools, and practices superficially, thus obtaining little, if any, of the intended benefit. In order to effectively reinforce behaviors, those behaviors have to already exist as part of your culture.

The Role of Culture

I've blogged multiple times about the difference between doing Agile things and "Being Agile" ("Agile: The Book vs. The Movie", "Understanding the 'Why?'", "Descriptive versus Prescriptive", "I Madge I'll - Agile Posers", etc). To deliver the best results possible, your team has to be Agile - they can't just superficially adopt an Agile implementation. When asked to explain Agile, they should go to the manifesto first and the day-to-day second. At the risk of getting too theological, Agility is a state of being and influences all aspects of a person's life (at least at work). Simply put, you have to establish a culture of Agility, where the principles are embedded in your company's DNA.

The same is true of Innovation or anything else. When people think of innovative companies they think of companies that have a culture of Innovation. Innovating isn't something you do as much as it's part of who you are. Of course their processes, tools, and practices reinforce Innovation, but that's a byproduct of something deeper.

As I said before, you can use tools from the Agile toolbox to help drive Innovation, but it can't just be lip service. If your company was built on Innovation (and it probably was) then get back to your roots and tell that story; remind people where they came from. Make Innovating seem natural, like coming home and eating comfort food.

No Silver Bullet

I'm going to be going more in-depth in the coming months on the idea that no one Agile implementation should be seen as a Silver Bullet. Before you implement anything, ask yourself "Why?"; what result or behavior are you hoping to get from it? Understand that different Agile processes, practices, and tools have their own strengths and weaknesses. Sometimes you'll have to blend different solutions to make something new that fits your needs. Sometimes you'll have to invent something that hasn't been done before. Most importantly, you will always have to examine what you're doing and ask what you need to change to be better. Keep the mantra "Never Perfect; Always Better".

Can Agile help drive Innovation? Sure. But some approaches will do that better than others. Begin with the end in mind, but remember that this is a journey and you don't know what you don't know. Be willing to adapt, but don't lose sight of your vision. If it sounds hard, that's because it is; if this were easy then we wouldn't be having this conversation!

What do you think?

Seriously, what do you think? Please tell me! I really don't like the "thought leader in a vacuum" setup. What I've shared here is the result of my learning and experience, but I recognize I don't know everything. I crave the perspectives of others. I want your insights, suggestions, criticisms, reinforcements - all of it. Leave a comment or tweet me @MLCarey321.

Monday, November 4, 2013

The Best Lesson I Learned the Hard Way

Let me preface this by saying I do not recommend learning lessons the hard way. There's a wealth of human experience out there to learn from before you go off and make the same bone-headed mistakes. However, bone-headed mistakes will be made regardless of how cautious you are. In those cases, please go into it with a Fail-Fast, Kaizen mindset so that the mistake becomes a short-lived learning experience.

The Almighty PM
I've made a lot of mistakes in life, which means I've learned a lot of lessons “the hard way”. The one such lesson that has had the most impact on my current work life took place my senior year of college. I was in a capstone class where we were to organize ourselves into a real software team to develop a real-life application on top of a new API that we had Beta access to. We were given a little guidance, including how previous classes and many companies organize themselves, but told we could do whatever we wanted. We ended up with a very “traditional” model of a few Dev teams, a few Testing teams, an Architect, and the one Project Manager to rule them all.

I was eager to prove myself, so I went for the PM position (oddly enough, I didn't really have any competition from my obviously more self-aware classmates). The problem is Computer Science majors in their fourth (okay sixth) year of study haven't usually had any PM classes or experience. My architect laid out the architecture, lots of open source Java, wikis, subversion, etc, and the class was excited to get started. However, I was concerned about how to manage the work, as well as expectations. I started demanding individual Gantt charts up-front for the duration of the semester. I was asking each team lead to compile the Gantt for their team and report that to me so I could put it all together.

The Blind Leading the Competent
Basically, I was trying to command-and-control manage a team of part-time team members, all of whom were taking at least 12 hours that semester and had limited time to spend on this class. Of that limited time, I was asking an unreasonable amount of time on administrative work. My architect was an excellent “Yes Man”, which made it difficult for me to see what I was doing wrong. The tipping point came when we weren't getting the information we were asking for from the teams and we were way behind where we expected to be on the application. Our professor was pulled away for a class and gave me the option of keeping the class time or giving everyone the day off. My architect decided, with my blessing, to use that time to express our mutual disappointment with the team and shame them into getting their act together.

I knew it was all over when the architect started his oratory with something to the effect of “By the time I finish talking you should all be mad at me, and that's okay because I'm mad at you.” It was a horrible hour and ended with our organization in shambles. I was a broken, broken man who was certain I was going to fail the class.

A Lesson in Servant Leadership
Then one of the testing team leads asked if he could meet with me to talk through things. He kindly and patiently explained why what we did was so stupid and counter-productive. He told me that the teams needed a cheerleader, not a dictator. He asked me to spend time with the individuals on the teams. He wanted me to spend more time in face-to-face conversations instead of mass emails. He wanted me to understand how hard the class was working to accomplish a common goal so that I would stop wasting their time with unnecessary busy work. He wanted me to focus on listening and removing roadblocks. He is now, unsurprisingly, a manager at Amazon, and someone I still hold in high regard.

I didn't fail the class because the objective of the class wasn't to develop a fixed scope within fixed time and budget. My wise professor wanted us to experience real-world project work as best we could in an academic setting and learn from it. He saw me try to turn things around at the end and knew that I had seen the error of my ways. He knew that I had learned my lesson, albeit the hard way.

Lesson Learned
So what lesson did I learn? To be honest, I initially thought that the lesson I was supposed to learn was that I suck at Project Management. I could not be trusted with leading a project to completion. I could code well and kinda sorta test, but the realm of Project Management was beyond my grasp.

This lesson was reinforced when I entered the workforce a few months later and saw command-and-control PMs leading projects “correctly”. These were professionals who did not go on rants against their developers and testers. They were not perfect, but they did their best to make things work in alignment with the command-and-control, iron triangle, Waterfall-driven style they were trained to use. Despite their best efforts, I found myself in War Rooms quite frequently, and my wife learned to not expect to see me in the months surrounding a Production install. I didn't realize it at the time, but I was living through the hell that I had just put my classmates through.

Of course, that wasn't the PM's fault. As I said, they were doing the best they could, and we were doing the best we could. The problems were much deeper than any one individual. We had to change the way we work, starting by not only recognizing but emphasizing the human element of software development.

The real lesson is that command-and-control Project Management doesn't work. Had I taken the time to understand Lean/Agile development processes and practices, I could have done a decent job as a Scrum Master or Agile Servant Leader in my capstone class. I might have even done some coding and testing alongside my cross-functional Agile development team(s). Even before I learned that lesson, I learned the importance of being a cheerleader. I have always striven to let the teams I've worked with know what a great job they're doing and how much I appreciate them. I haven't always succeeded, but that pep talk I received my final year of college never left me.

Once I realized that what I was experiencing in corporate life was essentially the same as that capstone class, I vowed to change things. I delved into the world of Lean and Agile and brought the things I learned back to my area. To many people it looks like I've picked it up extremely quickly; in reality, I've been learning this lesson for at least six years. If anything, I'm ashamed it took this long to finally “click”.