Experience report: Improving 300% by moving from Scrumish SAFe to Kanban

Written by

Many times we (as Agile coaches) quickly discover that the Agile frameworks used in our client organizations are deeply incompatible with…

  • The nature of their work 
  • The flow of their work
  • Their overall organizational context
  • And the list goes on!

This is especially challenging when the organization has chosen to quickly scale a single framework with brute force while using a one-size-fits-all approach without considering systemic root causes to their performance issues. In my experience, this often happens when management aims to address symptoms that manifest at their flight level for immediate pain relief and inadvertently neglect to probe more deeply into systemic causes. 

As a result, the teams doing the work struggle through these mandated ways of working that ultimately lead to more problems than they solve! 

This blog post is about my experience when Kanban was a better fit than Scrum, the SAFe caused more problems than it solved, and the organization was afraid to let go and let teams explore what Agile Ways-of-Working works best for them. 

I’ll be sharing pieces of the journey that led to a 300% improvement in Lead Time along with three key questions that can help keep you on track if you find yourself in a similar spot, which are as follows:

  • #1: What problem were you brought in to solve? And what Agile good practices could help?
  • #2: Is the problem actually the problem? Or is it a symptom of something else?
  • #3: If it is a problem, are the current Ways-of-Working helping or hindering? What does the nature and flow of work tell you? What about the organizational context?

This blog post is structured according to those questions where I share the success story that came from carefully answering each one. Hopefully the article will be helpful resources if you find yourself navigating through the same murky waters!

#1: What problem were you brought in to solve? And what Agile good practices could help?

Like most Agile Coaches, I started by seeking to understand the measurable problem I was brought in to solve. And because the client wanted quick results, I also looked for what Agile good practices might help to secure some early wins.

In the specific experience I’m drawing upon for this blog post, this was, “We need to improve this department’s predictability to at least 80%”. 

The landscape

The teams were using Scrum in a SAFe set-up based on a decision 5 years ago to centralize everything Agile under one big SAFe roof. Managers and stakeholders seemed to like using PI planning as a really large batch transfer that happened every 3 months. The outcome amounted to quarterly Gantt charts from each team detailing at a User-Story level how they intended to work on the massive influx of work items the business pushed onto them to complete. Although stakeholders actively invited the Product Managers and Product Owners “to say no” to taking on more work then they could handle, during my interviews I learned that the unspoken cultural norms made it very unsafe to do so. 

What the data showed

Team interviews and high-level value stream maps indicated that, like most organizations, the teams were flooded with work due to a push-based system, features were bundled into projects creating very large batches, and there was a lack of prioritization due to minimal program or even backlog management. The result was a push-based system that was “going through the motions” of Lean Agile but not experiencing the benefits.

Although Scrum is a pull-based system, it wasn’t being used in that way. To get the system heading back into the right “pull-based” direction, I recommended we start with some known good practices for balancing capacity with demand and optimizing flow, such as unbundling features from projects to work in smaller batches, practicing regular backlog refinement, implementing clear definitions of ready and done, and implementing some good program management practices that include prioritization. 

I also recommended that we work on creating T-Shaped team members so work wouldn’t grind to a halt if people were sick, on holiday, or chose to leave the team.

T-shape that describes how to form teams consisting of members with expert knowledge together with members with good enough knowledge of the issue.
T-shape that describes how to form teams consisting of members with expert knowledge together with members with good enough knowledge of the issue.

What I did not expect

Although the Agile Coach and Product leadership team agreed with my assessment and recommendations, which I expected, the team-of-teams did not! They were firm on their stance that adopting such practices would not address the root cause to their pain points. Moreover, they disagreed with management’s assessment that they have a predictability problem. 

As a coach, this was interesting territory. I was brought into drive change in a team that believes no change is needed and doesn’t agree with the measures used to gauge their performance. 

After a few honest discussions with the team, I found myself asking the following question:

#2: Is the stated problem actually the problem? Or is it a symptom of something else?

I began to question various assumptions that were in play, such as:

  • The assumption that the team-of-teams had a predictability problem
  • The assumption that measuring predictability according to PIs is meaningful
  • The assumption that optimizing around SAFe’s PI planning would help predictability
  • The assumption that Scrum is the best way-of-working for the teams

I also went “pull-based” as a coach, which meant I chose to only initiate those Agile good practices that the teams pulled from me (as opposed to what the other coaches aligned upon as “good for the teams”). 

Uncovering the real problem

What I found was that:

1. The way teams were working and being measured were deeply incompatible with the flow and nature of their work, and
2. The teams were longing for a new Way-of-Working. 

Much of their work had unexpected delays and ad-hoc requests that were due to the nature of the projects in which they play a small role, as well as unavoidable support work. 

Moreover, they often lacked required up-front information at the start of a PI to gauge whether or not they could complete the work during the PI. This means a lot of built-in variability in their flow of work plus a chronic “falling short” of PI predictability standards, given the lack of data they had with which to plan. 

I also learned that the choice of Scrum was made by the Agile organization without accounting for the nature and type of work being done, the ultimate value meant to be delivered, or the organizational context (nevermind the team’s preference). Nor were there feedback loops in place or sufficient empowerment of teams by the organization for the system itself to evolve into a lean, value-driven system. 

The most interesting discovery was that stakeholders outside of IT ultimately didn’t care about PI predictability. They were more concerned about whether or not the teams were delivering according to their project milestones, which were anchored to projects that live outside of IT. So a team’s “predictability” according to IT’s arbitrary 3-month PI planning cadence had no business value. What the stakeholders did like, however, was the really large batch transfer of work items. As long as they got to transfer work items into IT every 3 months and that progress went according to their project milestones, they were happy.

Based on these discoveries, my conclusion was that the stated problem may actually not be a problem at all, It may be a situation created through metrics that don’t have business value and Ways-of-Working that don’t suit the needs of the team!

#3: If it is a problem, are the current Ways-of-Working helping or hindering ? Or is it a symptom of something else?

Working closely with the team’s Product Owners and Scrum Masters, it became increasingly clear that predictability, according to the way it was being measured, was not the actual problem. It was the Scrum-based SAFe Ways-of-Working that was the actual problem. 

PI plannings were nothing more than a replenishment meeting happening every 3-months where the items chosen were simply based on the odds of (1) there being enough information to start work, and (2) there being no upstream delays. The teams were involved in collectively hundreds of hours of Agile Scrum and PI planning meetings every 3-months just to place a bet on which epics they think would “win”. 

Moreover, a good portion of the work were ad hoc demands that created a great deal of variability in the flow of work. And given that business stakeholders (end users) are more concerned about delivering according to product milestones than arbitrary PI timeboxes, it also became clear that a measure such as “%OTD” (On Time Delivery) along with cycle time in a Kanban system would be a more accurate and meaningful measure that would also accommodate such variability. 

Making the case for Kanban

Visualizing these conclusions provided the business case we needed to pilot Kanban in one of their teams to prove that upfront-PI planning is not needed to ensure work is delivered according to project milestones. We held a 10-hour STATIK workshop where we played some Kanban simulation games and used the learnings to draft a Kanban board, team policies, and other Kanban artifacts. 

The result was astounding. The team met daily for replenishment meetings, weekly for retrospectives, and were available for dependency management during the PI plannings, but nothing more. And within three months, their lead time decreased from 24 days down to 8, which is a 300% improvement. 

It wasn’t long before other teams wanted to transition to Kanban! 

Conclusion

Taking time to question whether or not your team’s current Ways-of-Working are helping or hindering the actual problem you’re aiming to solve can perhaps help you to get the performance and predictability breakthrough your team(s) need. You may even find yourself adopting Kanban along the way and experiencing a 300% improvement, or better!

Shopping basket
Related Trainings
Our Trainings
Förändringsledning och ledarskapsagilitet – Nyckeln till lyckade techinvesteringar
Target Group: För dig som är nyfiken på att utveckla ditt ledarskap och förmåga att leda större lyckade förändringar och komplexa leveranser av ny teknik.
Nästa datum: 4 Februari, 2025