Sunday, February 14, 2010

It's time for Scrum to evolve

This week has seen a flurry of activity online concerning the current state of scrum, it's flaws, and if it should be changed.

While criticism online of a particular method is nothing new, more attention has been paid this week when Uncle Bob listed 8 issues that he had with scrum...

1. No technical practices. Scrum is great at giving project management advice,
but provides no technical help for the developer. Any good implementation of
Scrum needs to borrow technical practices from some other method like XP. The
suite of technical practices that should be added probably include: TDD,
Continuous Integration, Acceptance Testing, Pair Programming, Refactoring.

2. 30 day sprints are too long. Most scrum teams have either shrunk them to 2
weeks or perform some kind of midpoint check at the two week mark. I know of
some teams that have two 2-week "iterations" inside a single 4-week "sprint".
The difference being that they use the sprint for reporting upwards, but use the
iterations for internal feedback and control.

3. The tendency of the scrum master to arrogate project management powers. This
is not a problem with Scrum out of the box so much as it is a problem with the
way scrum sometimes evolves. Perhaps it is related to the unfortunate use of
the word "master". Perhaps the XP term "Coach" might be a better word to use.
In any case, good implementation of scrum do not necessarily correlate scrum
masters and project managers.

4. The C in CSM is unfortunate. Again, this is not so much about scrum out of
the box as it is about the scrum community. That letter C has gotten far too
significant for it's intention. It is true that the people in a scrum team need
to be trained. One of the things they should be trained about is the role of
the scrum master. The problem with the C is that it changes the notion of scrum
master from a role into a person. It is the person who has the C. In an ideal
case, the members of the scrum team will rotate through the scrum master role
the same way the members of an XP team rotate through the coach role. This
rotation is never perfect, and sometimes the role sticks to one or two people
more than others. But the idea was never to raise up a particular person with a
rank. We never wanted that C emblazoned on their chests.

5. Scrum provides insufficient guidance regarding the structure of the backlog.
We've learned, over the years, that backlogs are hierarchical entities
consisting of epics, themes, stories, etc. We've learned how to estimate them
statistically. We've learned how and when to break the higher level entities
down into lower level entities. Epics->Themes->Stories->Tasks.

6. Scrum carries an anti-management undercurrent that is counter-productive.
Scrum over-emphasizes the role of the team as self-managing. Self-organizing
and self-managing teams are a good thing. But there is a limit to how much a
team can self-X. Teams still need to be managed by someone who is responsible
to the business. Scrum does not describe this with enough balance.

7. Automated Testing. Although this could be considered a derivative of point
1, I thought it worth calling out as a separate point because it is so
fundamental. Scrum doesn't mention this, yet it is the foundation of every
agile effort. Agile teams work in short cycles because feedback only works well
in short cycles. But short cycles aren't enough. You also need objective
measurement of progress. The most reliable way to know how much a team has
gotten done is to run automated tests and count the tests that pass.

8. Multiple teams. Scrum has little to say about the coordination of multiple
teams. This is not a failing unique to scrum. Agile itself is virtually silent
on this issue. Scrum talked about the vague notion of a "Scrum of Scrums" but
that idea really hasn't played out all that well. Scrum-in-the-large remains in
the domain of certain consultants who claim to have an answer. There is no real
consensus on the issue.

Agile blogger Mike Cottemeyer have waded in stating that the concept of an Agile PO needs to be significantly rethought, and Alan Shalloway of Net Objectives has long maintained that Scrum (and other agile approaches) have long suffered from it's overty developer focus, and anti management bias. Alan has stated that extending scrum with lean/kanban can go a long way to make scrum significantly mire successful.

Jurgen Appelo has come in swingng to the defense of scrum, saying that a lack of technical practices is a strength, not a weakness as it allows scrum to be applied to other professions beside development.

For my part I am firmly with the critics on this one. Let me start by saying that I think scrum has some great things in it. The management practices are a good way for teams to get started down the road to becoming more agile. Simple things line retrospectives and standups reallly help noobs understand what agile is.

But that's is where it ends. Scrum is so simple and incomplete that it does not adequately serve it's target audience appropriately. It's fine to say that scrum is meant to serve multiple audiences, which is great in theory, but let's look at reality, the vast majority of scrum users are developing software. They are not graphic designers or painters or management consultants. I've seen teams get into serious trouble with flaccid scrum, ie no technical practices, because they thought they had what they needed. In reality they were worse off than the previous waterfall state. No documentation, external qa, but no CI or TDD, what a mess.

Scrum practices also come off as being a little naive and almost condecending to teams with even a little emotional intelligence. Rules like chickens can't speak (ie managers) at certain meetings and the notion that developers only have to deal with product owners seem like an extension of many of the spoilt developer attitudes that came out of the dot com era.

I actually think for agile to be successful teams need to take scrum practices and mature towards Lean an kanban. The language is more mature, and you can use more sophisticated but still simple tools to better model and manage more complex situations than one team serving one product owner.

Finally I think for such a simple approach scrum is a little dogmatic about what you have to do. Again this has to do with maturity, beginners might need to be told exactly what to do and who can talk in a stand up meeting, but teams that move a little bit up the maturity curve quickly find this kind of advice pedantic.

I won't even get into the whole certification fiasco, which to me is a complete joke.

So in conclusion I think owners of scrum need to take a look at this criticism and seriously consider how they can incorperate public feedback into improving scrum. This is critical if we are to continue to take scrum seriously.

18 comments:

  1. Great post. I've been addressing the limits of Scrum for quite a while. Moreover, I don't believe in user stories. They're too ad-hoc, and low structured to serve as a good unit of work and requirements for larger project e.g. a service oriented projects with five back-end systems, two front end portals and three external parties (like the project I was in recently)...

    Sander Hoogendoorn
    Principal Technology Officer Capgemini

    ReplyDelete
  2. Scrum is not perfect, but I've seen it work enormous wonders in several product development companies. You can critique Scrum all you want, but denying the success it had makes your story unbalanced. Of course in some cases you're right. Scrum could be totally wrong in some situations or needs tweaking on others, but sometimes it's spot on.

    You seem to propose an evolved Scrum (let's say Scrum 2.0) that includes more practices, has shorter iterations, less anti-mgt etc. But you will end up with a new method, with new issues. This Scrum 2.0 will be perfect for some situations, but horrible in places where perhaps Scrum 1.0 could have been perfect. The problem is a linear view on methods, where an evolved (newer) method is better in all respects than the previous one. This is rarely the case.

    ReplyDelete
  3. Sander,

    I completely agree, user stories alone dont cut it for most situations, especially when you have multiple systems (and hence multiple teams) depending on each other, user stories are mho one of te least valuable things to come out of agile, they are kind of embarrasing actually.

    Machiel,

    i do propose that scrum evolve, this is not to say that scrum hasnt provided value, the point is that any method needs to evolve or time to or its value diminishes, lets face facts, a static set of practices is not agile, its the opposite of agile. Not all change will result in progress but if we dont have the courage to continue to inspect and adapt our methods then why are we talking about embracing change in the first place, seems like a contradiction to me.

    ReplyDelete
  4. Maybe I am naive, but ain't the Scrum team "empowered" to decide for themselves on issues 1, 2, 5, 7 through retrospectives? Some guidance might be useful to start with, but setting technical practices as fixed part of process might be counter-productive.

    ReplyDelete
  5. I agree Premek and have seen those practices evolve when the team is challenged appropriately with the problem.

    Simple is smart. Simplistic is naive and short sighted. Simple has been an enginerriing maxim for a long time. If we are to talk about coding practices then the engineering must evolve from KISS.

    - Doug
    http://doug-shimp.net

    ReplyDelete
  6. The minute you bolt tech practices into Scrum, fix the sprint length to some convenient number and heap on process for things like backlog management will be the day Scrum stops being agile, and the first day of its trek to the methodology garbage can. Its precisely because it has these "flaws" that it works so well.

    If you need more - use it with XP and make sure your team inspects and adapts in everything, not just the code base.

    ReplyDelete
  7. Premek,

    Practices don't need to be fixed technical or otherwise, but IMHO they should be mentioned, short iterations become really hard to sustain without technical practices, not mentioning them, or not having an explicit mencanism to plug them in ( which I believe Ken Shwaber is doing right now) seems like a cop out.

    Doug
    simple is good and simplistic is bad, you couldn't have said it better.
    IMHO scrum is simplistic, scrum of scrums, what I did yesterday and today, the default planning boards? One PO? Come on...

    Interesting projects need more than this...

    Mike
    I don't get the reluctance to adopt both technical practices, even something like automate repetitive work with some examples (ie builds and tests) would go a long away.

    The real issue is the number of organizations who get all obsessed with adopting scrum, on it's own. The problem is you can't use it on it's own, which limits it's usefulness. it always required significant extension and worse significant alteration. I'm ok with teams adapting to suit their needs, it's what they should do, but I think scrum should provide a better starting place than it does.

    The lack of tech practices cause me to think it's naive as a process, but the "pigs can talk here and chickens can't" and all the retoric around "if you don't do this you aren't doing scrum" bothers me alot more. That and the certifications.

    For all that it's still miles ahead of the waterfall stuff I see so many clients do.

    ReplyDelete
  8. Thank you Jeff, this post is like a breath of fresh air for me. I am one of the many Scrum practicioners who are happily doing 'scrum but' for many of the reasons you mention. It bothers me that so many in the scrum community seem so inflexible and unwilling to inspect their processes, and try figure out why they don't work everywhere.
    I am a Scrum advocate, but that doesn't mean I can't challenge the process

    ReplyDelete
  9. Greg,

    Thanks. This post drew alot of criticsm, what some fail to understad is that I use and reccomend using alot of scrum practices. There's lots of good stuff there. But scrum is just to fixed and to incomplete. Practicioners are always reinventing the wheel to make it useful.

    I think consultants want to protect the brand and the simple 2 days uber expensive certification scam. It's causing bad blood in the agile community IMHO

    ReplyDelete
  10. Jeff,
    Good blog although I don't think I agree. I think Scrum is a simple set of practices that are very difficult to actually implement. They say it takes 2 years to really get Scrum down and that is with a consistent team. I think there is one of the biggest challenges.

    I think User Stories become a great tool if you have a team of developers with domain knowledge. It can be a major problem if the domain is not there AND the PO is not active.

    My biggest problem with using Scrum were:
    * Definition of 'Done'
    * A PO that was not involved enough
    * Balancing the team velocity with what management wanted.

    I think Scrum works as advertises and I would not change anything about it. It does not promise immediate improvements in process and development, but it does shine the light on problem areas.

    (sorry if my formatting is bad, my laptop is letting me down)

    ReplyDelete
  11. Hi Jeff,

    >> So in conclusion I think owners of scrum need to take a look at this criticism ...

    I would phrase this differently. So in conclusion I think the people responsible for software development projects need to take a look at this criticism and see whether it applies to them.

    I'm not a big supporter of Scrum and if I am looking for a software development methodology then I would choose one like XP. Scrum is not a software development methodology.

    And here lies the problem. People want others to tell them what to do and make their decisions for them. Scrum doesn't do this, it says here are some principles and guidelines and work the rest out for yourself.

    I agree that this message gets lost in the noise the marketing, and the CSM training :), but that's the problem.

    Forget the slogans, the hype, and false certifications and read the source material for yourself. There is a very good book on Scrum (Beedle and Schwaber), and other good Agile primers (Alistair Cockburns Agile Software Development comes to mind) that place Scrum in the right context, and then decide what you need to do, and what will work best for you.

    Scrum doesn't claim to be a universal solution for everyone out the box. In fact no such thing exists, even XP :)

    (BTW. XP is a great place to start if you plan to produce software :))

    When will people get it, that they need to think for themselves and stop waiting for others to do their thinking for them?

    Paul.

    ReplyDelete
  12. Hi Jeff,

    Just to be clear Kanban is not a software development methodology, neither is so called "Lean Software Development". None of these things tell you how to produce software.

    XP does, but no one wants to do XP. Go figure :)

    We need to stop blaming Scrum and start looking at ourselves. Otherwise we will just repeat the mistakes of the past all over again.

    Lean Software Development Certification anyone?

    Paul.

    ReplyDelete
  13. As a Scrum practitioner for two years now, I read your blog and agree with Michail that you do not speak at all to the phenomenal successes it has brought to organizations that fully adopt it. Instead you restate criticisms that are found all over the net (not much originality here Jeff). Scrum has been a major factor in our ability to deliver near product ready solutions every TWO weeks.

    Bob Contreras

    ReplyDelete
  14. I actually enjoyed reading through this posting.Many thanks.

    ReplyDelete
  15. I actually enjoyed reading through this posting.Many thanks.

    ReplyDelete
  16. This comment has been removed by the author.

    ReplyDelete
  17. It's probably true that Scrum (and XP) should either evolve or go away.

    My hunch however is that Uncle Bob is merely unhappy that scrum is outpacing XP in the mindshare war.

    Uncle Bob says that Scrum doesn't have technical practices, but neither did Waterfall.

    More commentary at my blog url

    ReplyDelete
  18. ...change is constant.

    Discussions like these is just another proof of what the future will bring;

    - Scrum will change
    - Scrum spinoffs

    My prediction; time will show us that the scrum framwork will only include very small changes, and this will force the birth of several new agile techniques based upon scrum (spinoffs), but with less/more/other additions supporting gained learnings and belifs.

    Maybe the biggest challenge in the future will be to find the right agile technique amongst the many?

    ReplyDelete