Thursday, December 8, 2011

Five Step Illustrated Guide to Setup a Kanban System in an Enterprise Organization

We've been through a significant number of Kanban implementations and our approach has largely been based on the approach outlined in David J. Andersons Kanban book. During our travels, we find that we sometimes need to make certain adjustments to the standard steps and during our recent work with a large-scale organizational transformation where we are applying Kanban across the entire organization of 200+ people we have encountered some situations that you only run into when you are in a larger enterprise setting. I thought it would be useful to others for me to provide a real-world step by step illustration of how we have been approaching the problem. If your about to kick-off a Kanban adoption in an enterprise IT organization or in the midst of one and struggling, you may find this useful. It's a simple 5 step approach that has always produced good outcomes for us while respecting the pace of change a typical IT organization can absorb.

Step 1) Identify the various upstream and downstream groups with a special emphasis on external "support groups" for the group adopting Kanban

In our scenario, we were helping an IT Development Group adopt Kanban. To start, we need to first identify who are upstream parties that give us work, in this it's the clients/business themselves that have a PM/BA function and there are 5 of them. There is one downstream group, the Release Support Group that is responsible for packaging, and pushing code into staging and production. The final group to look at is other support groups which tends to be a key consideration in enterprise settings since most IT organizations have moved towards functional consolidation. For this particular group, they rely on a DBA Support Group, and a Testing Support Group in order for them to complete their work and pass it to the release group.

Step 2) Identify the work types that come in from the upstream provider and work with the IT Development Group to agree on a set of standard work types to setup a "Work Type Filter"

One of the lessons we learned during our work with a number of teams is the importance to treat the work type analysis as a capability and not a one-time event. Work type requests evolve over time and the team needs to develop the capability to conduct this activity and evolve a standard set of work types. I find thinking of this as implementing a Work Type Filter for the team helpful so they understand it's a capability and mechanism they need to operationalize. Work type analysis tends to be always be an interesting conversation with the IT Development Group in enterprise IT settings since the development teams will have other "responsibilities" besides writing code. They are often called upon by the client or other groups to provide two types of non-development services, Consulting, and Vendor Support / Management. Since functional consolidation tends to exist in most IT organizations I have seen, some type of centralized PM/BA function is often responsible for gathering requirements and providing estimates and plans. These groups lack the technical knowledge to complete these activities and as a result seek out the development groups for help. The second type of non-development request is often Vendor Support / Management. Business demand for solutions often tends to be non-stable and peaks near the end of the annual budgeting cycle as they feel the pressure to spend their budget which causes demand to exceed internal development capacity which in turn leads to the organization running to scale up with external vendors to take on the extra work. This will generate some work for the development group as the PMs will often ask the group to provide technical oversight or support to guide design, coding and help with code integration and promotions. I find it extremely helpful to separate these non-development requests from development work types since they often cause noise in the system and impact capacity in a different way. Turning back to our real-world example, here's what we ended up with:

Step 3) Map out the internal value stream, and identify where hand-offs and coordination with support groups occur in the process

Now that we understand the boundaries of our Kanban system we can start digging into the internal workings and understand how everything connects. Value stream mapping is often a useful tool for this, but I find keeping it simple works best. The important part of this step is to explicitly map out where the "interface" to the team is with other support groups and understand what information / value / artifacts is traded (i.e. what the interface contract should look like).

Step 4) Visualize all work in terms of the standard work types, let the system run to understand throughput and then set WIP limits with an emphasis on the input queue and external support group columns

At the end of step 3, we should have a kanban board and we can start onboarding all the work the IT Development Group is currently working on in terms of the standard work types we defined for the Work Type Filter. I find many teams struggle with setting WIP limits right away, and they need to use the Kanban system for 2-4 weeks before they have the right comfort level to have this discussion. At that point, the throughput metric should be tracked as this will provide a helpful frame of reference to set the initial guesstimate for the WIP limits. The key columns I focus on with the group is the input queue and external hand-off columns. The rationale behind focusing on these columns is they are the interfaces to the outside world and we want to start setting some expectations and commitments around these as the IT Development Group needs their help to deliver work. The input queue keeps them fed and the hand-off columns keep them running. I often leverage the throughput metric to orient the team around a WIP limit for the input queue based on the simple guideline of only take in what you can finish.

Step 5) Expose the queues to the clients and help setup and operationalize a prioritization framework as a Prioritization Filter

At this point, we expose the queues to the clients and start pushing prioritization of work to the customer. We also find in enterprise IT settings, providing only a next/this month queue is insufficient for their planning process. As a result we often provide a prioritized backlog mechanism with two replenishment cadences, an annual/quarterly one for strategic/budget planning a monthly one for just-in-time decisions. This is where we often see the most churn and is typically the most challenging part of the Kanban process. Many IT organizations are great at prioritizing within one group, but across organizational groups is a struggle despite attempts at elaborate and agonizing prioritization committees/boards. To help with this problem, the Kanban system arms us with two pieces of information, a common unit of currency (the standard work types) and delivery throughput. Leveraging these two pieces of information, we recommend working with the clients to analyze all their known demand, break it down using the Work Type Filter into our standard work types, and then calculating the gap between demand vs capacity in terms of throughput. Once the gap is known, there are two options, scaling up with more external vendors or prioritizing the work. In this scenario, scaling up was not an option due to the tight deadlines and the ramp up time needed to onboard new resources so we had to go with the prioritization option. To help with the prioritization problem, we recommend holding a workshop to align the requests with strategic priorities for the organization and then applying a FIFO policy to these requests while considering true fixed date items. This provides a workable interim solution as it forces the clients to focus on today with FIFO. The goal is to get to a set of demand channels based on what's ready to go now or soon and set % allocation against these channels. We then provide the client an opportunity to adjust the % allocation during the monthly replenishment meetings.

After this step, you should have a fully operational Kanban system. Now the hard but fun part remains which is evolving it and optimizing it for performance.

Monday, December 5, 2011

Looking at Why Agile Works Tells You How To Make It Scale

Recently, I have found myself engaging in conversations with clients who are new to adopting agile and are struggling with how to get started, especially in a large-scale, enterprise context.

I don't find myself talking about which agile method they should adopt. Rather I focus on why agile works.

The reasons are simple, but that doesn't stop the principles from being highly disruptive to the waterfall command-and-control way of thinking. Even adoption of simple principles requires a radically different mindset.

 IT organizations don't have to adopt agile methods whole to start using these principles. sure agile methods are battle tested, and you work in particular contexts, but adoption can seem overwhelming.

I encourage clients who aren't willing to take the agile plunge to take a look at some of the simple but disruptive principles and see where they can introduce one or more to their organization. 
  1. Replace Stage Gates with Regularly Scheduled Cadence
  2. Describe Planning Units As Increments of Business Value, And Treat Them As Atomic Units That Can Be Designed and Delivered Independently
  3. Limit Consumption of Demand According to the Capacity of Workers to Satisfy That Demand
  4. Use Visualization to Make Sure That Everyone Understands What the Processes Are, and Where the Current Work Fits into That Process
  5. Invest Both Time and Capital To Improve the Way Work Is Completed

Replace Stage Gates with Regularly Scheduled Cadence

Stage gates are the familiar command-and-control waterfall tool of choice of to ensure high quality and stable delivery. Work cannot proceed until the entire batch is deemed to be of sufficient quality to proceed to the next stage. This in theory prevents defective work from proceeding to the next stage, and helps mitigate delivery risk.

The problem with stage gates is that they actually reduce the amount of quality reviews and inspection done for projects that really require it.

Take 2 programs, the first program contains healthy projects, the second one unhealthy project.

In the first program projects produce good quality work, projects almost always made it clear to stage gates on time, and pass on first review. This means that the stage gate reviews happens relatively frequently across the program.

Now let's take the second program, unhealthy work causes the projects to hit delays. These delays in turn cause stage gate reviews to be delayed for the projects within this program. This means that stage gate checks for the program actually happen less often.

Any process that mandates less checks for an unhealthy project versus more check for a healthy projectis inherently flawed. 

The answer to this is favoring cadence over a state gate approach. A cadence is a regularly occurring event, and is at the heart of the iterative model inherent to agile methodologies like scrum and extreme programming.

Transforming a waterfall process into an iterative one end to end may be more disruption than many IT organizations can bear.  

Rather than trying to reengineer the entire process, a simple fix is to mandate that all work in process within a particular stage should be reviewed at a regular cadence by required governance folks and/or downstream knowledge workers. 

Eventually knowledge workers can start categorizing work into  two clumps, work that can be moved into the next stage, versus work that is not ready to move into the next stage. 

Suddenly knowledge workers come to the realization that work that is ready can proceed to the next stage can do so, leading unready work behind.

Putting regularly scheduled cadence reviews can be the first step towards getting to to smaller batches of work. 

Whenever I see any stage gate process in a delivery model, I tend to recommend leaving it relatively intact,except that I replaced the stage date with a cadence review.

Describe Planning Units As Increments of Business Value, And Treat Them As Atomic Units That Can Be Designed and Delivered Independently

Agile methods force teams to look at planning from a different perspective. You are expected to create plans out of small, stable, atomic increments of business value. 

  • Being small means work can be effectively estimated. 
  • Having business value means that completing work is a great way to measure progress 
  • Being atomic means that work can be built and given to the client by itself, you can get toto business value sooner.

Most agile methods  express this concept in user stories, I personally like the Kanban approach of taking the types of work that are already recognized by the delivery organization, and gradually coaching teams towards expressing these in smaller, atomic units of work

As stated above, creating small atomic units of work facilitates movement of those pieces independently of each other, this makes iterative and/or flow-based development much easier.

Limit Consumption of Demand According to the Capacity of Workers to Satisfy That Demand
Agile teams that ignore this principle are not as successful as they could be. Even very effective agile teams run into trouble when they have a very large backlog of work.

If the release plan contains over 2 years of stories, chances are that there is enough upstream churn to affect the overall success of the agile team. Regardless of how good their velocity or quality is. 
Getting to business agility requires that everyone asks the question "when can I actually complete this" when considering looking at new projects.

Getting management, stakeholders, and even knowledge workers to understand the toxic nature of inventory can really be a challenge. One that I struggle with every day.

Use Visualization to Make Sure That Everyone Understands What the Processes Are, and Where the Current Work Fits into That Process

Fog of war is probably the most serious impediment to effective knowledge work. I don't think fog of war requires a large team to take effect.

I find after a team size of 2 to 3 people it becomes challenging to understand what other people are doing to the point where it interferes with good teamwork. 

Agile planning boards, Kanban systems, and other information radiators are an excellent example of how to combat this fog of war. 

An entire agile method does not need to be adopted to start taking advantage of these information radiators.

Invest Both Time and Capital To Improve the Way Work Is Completed

While I have stated this one last, it's probably one of the most important if not the most important of these principles. This is something I continue to struggle with both for my own work, as well as how to get teams that I coach to effectively do this.

Of course agile and lean methods have plenty of tools to address this, from retrospectives, to operational reviews and the A3 process. 

Different approaches can be used, such as explicit periods of time for improvement events, a minimum inventory level for improvement were tickets, or a commitment to complete all improvement activities after they are raised. 

What is important is that taking time to improve performance should not only be tolerated, but mandated. 

The point here is that agile organizations are adaptive, they continually change the way they work, because the investment always more than pays off in the value produced. 

I'm sure I've missed a few important principles, and I'll add to this list as I ago. Everyone out there feel free to contribute.