Performance Is Not Improving Close to What Is Expected
Charlie coaches the Product Owner to do some metric oriented root cause analysis, and shares this analysis with the rest of the team to get their thoughts. The bottom line is that while throughput and quality are improving, they are not receiving the magnitude of order improvements necessary to keep the backlog from growing at a rapid rate. There's simply too much demand coming into the system.
There Are Still A Lot Of Production Level Defects, but Code Quality Is High
Charlie thinks it's a good idea to take a closer look at the source of all of the failure intake coming into the system. Despite the team having adopted a better UAT, production level tickets continue to be a major source of demand for the team, consuming a lot of their effort. Having spent a good deal of time with this team, Charlie knows that their quality is actually quite good, and no longer believe that this is the source of production problems. Charlie marks this ticket within the Urgency section as yellow, meaning that more research is required.
Production Bugs Are Being Used to Capture Requirements
After a couple of facilitated workshop sessions where defects are analyzed, a number of common causes become clear. Many of the defects are actually uncaptured requirements. There appears to be a whole category of requirements that were not captured at the beginning of the project, and as a result the solution design needs to be re-factored to consider these features. The current practice in dealing with these requirements is for the operational experts to capture these missed requirements as production level defects, the team then attempts to fix the system in an ad hoc manner to meet these new requirements. No one is looking at these requirements from a holistic perspective, to determine overall impact on the system, or whether any reflector and is required. The result is the system never quite satisfies this category of requirements.
Story Mapping to the Rescue
Charlie discusses this with the application maintenance solution team and suggests that they bring back in another concept that was also retired from the change canvas, namely story analysis. If you remember, Charlie had removed this as a concept from the Target State along with other aspects of agile practices. His change recipients have felt at the time that this type of analysis was to heavyweight for maintenance type activities. Charlie suggests that they bring in a technique known as story mapping, as well as some other story analysis techniques such as playing poker, to help group defects into similar themes, and restructure them into one or more common user stories that can then be estimated, implemented and validated.
Charlie is now able to modify the Benefit section of the change canvas to have a much more modest improvement of throughput, as throughput is no longer the issue. Implementing these requirements now is, so with this in mind he adds a new Benefit; to reduce the gap between requirements and what has been implemented in the system. He wants to reduce this gap to one third of its existing size within three months.
Success criteria is likewise changed, throughput metrics are now much more modest. More critically, a new success criteria has been added that will measure the team's ability to analyze requirements and get them validated by the business at sufficient speed to reduce the backlog of unmet requirements. Initial suggestions are that if the team is able to process six story "themes" per month, then they will be at the right velocity.
Charlie also updates the Commitments to reflect both training as well as continued effort to perform this story analysis, something that will become a major responsibility for the solution analysts, as well as a commitment to the product owners well. Coaching has also grown to a full-time responsibility for Charlie, he's now helping on a number of different fronts but so far is getting good return on the time he spent with this team.
Charlie updates the Action section to reflect the new coaching that he will perform related to story mapping and estimating.
Charlie and the team agree that the Vision statement should now reflect how the team is working to evolve both the process and the underlying technology system to meet the needs of the business, both in terms of growing demand as well as correct functionality.
Once the canvas has been updated, Charlie introduces a new improvement experiment to validate these changes.
Check out the Rest of Lean Change - Chapter 5: Walking through the Validated Change Lifecycle with a Comprehensive Example