Agile Backlogs. Sigh.

Frederick Taylor Responsible for bad Agile Backlogs

Frederick Taylor, the man most responsible for the terrible state of most Agile backlogs.

What a mess Frederick Taylor created.

We can’t blame him completely, of course. He was just an engineer born in the mid 1800s who asked a simple question: “Why don’t we break up industrial processes into smaller and smaller pieces and then use the scientific method to improve on them?”

This seemed like a completely natural idea, and it’s done wonders for a lot of manufacturing, but it has also led to the oppression and murder of millions. Perhaps even the sorry state of your current agile project.

The result of Taylor’s question was “Taylorism“, or Scientific Management.

Lenin was a big fan. So was Stalin. It was used extensively in the old Soviet Union to measure — and punish — workers. It puts forth a way of thinking that feels like common sense. Even today, methodologies like Six Sigma are built on defining processes, measuring them, and so forth. The assumption is that all processes are naturally decomposable.

Who can disagree? As technologists, we learn to create technical things by breaking a big problem into smaller problems, solving the small problems, then reassembling the whole. We solve problems by scientific deduction. It only makes sense to use these tools wherever we can.

The problem is that they don’t always work.

Our Misconceptions About Agile Backlogs Are Killing Us

First, backlogs are not requirements. Say this over and over again until it finally sinks in. Agile backlogs are a symbolic work token list. The tokens represent conversations, diagrams, requirements, tests, and all sorts of other “real” work that occurs later in the project. They’re just tokens. Yes, they work best when you make the titles in such a format as to describe the future state of the system “When the tornado hits, as a farmer I need someplace to hide so that the tornado will not kill me and my family” But simply because you are using the old trigger, actor, verb-noun-phrase, goal pattern of requirements-writing doesn’t mean that you are writing requirements. It just means you are stating your stories as future tests, which is a really good thing to do. Always start with tests!

This means that breaking them up into smaller pieces, like you would do a programming problem, is not appropriate. You break them up at the last responsible moment to allocate and plan work, not to decompose the system.

Enough Detail

Enough with the detail! Enough already.

Second, people can only handle so much. If you’ve ever sat through a backlog session where somebody brings up a 300-item list on a spreadsheet and then the team plods though it, you know exactly what I mean: there is a natural limit to how much I can naturally work with. The team is not full of robots. The backlog doesn’t work as a symbolic work-token list if the people involved can’t mentally keep track of the symbols in the list. The beauty of backlogs is that you can create the symbols at any level of abstraction you like. Hell, you could have one backlog item, “Do Stuff”. It would be worthless for helping plan the future activities of the team, but it would be much less worthless than the huge monstrosities many teams are using to do the same thing. Big fuzzy backlogs are easily broken up and further defined as needed. Huge, finely-detailed backlogs bring cognitive death to anything they touch.

Third, by focusing on Agile backlogs you only hit one part of describing the work. Even if you get the idea that backlog items are simply abstract symbolic work tokens, many times you still start thinking that the future behavior of the system is all that there is to represent the system ahead of time. This is simply not true. There are three kinds of information the team works with to create a solution, and only three: information about behavior, information about structure, and general rules that must be applied in various circumstances. That’s it. What I find is that if you’re in a room with a lot of architects, they’ve got structure nailed — lots of diagrams and technical sketches. Sometimes too much. If you’re in a room with a lot of folks that used to be Business Analysts, they’re really good at talking about future behavior. Sometimes too much. Oddly enough, nobody cares much about broad rules, stuff like “We need all the screens in Klingon too.” Go figure. Perhaps if you were in a room with compliance officers, they would be really exact about all the business and non-functional rules that must be satisfied. Don’t know.

Future behavior is the one that gets the most attention from our Product Owners and Managers, because that’s what makes the most sense to talk about. What will the system do next week once we’re done? It’s a natural form of test that anybody can understand: what did you say the future behavior would be? Is that true today? That’s why the backlog is stated in terms of future behavior. But you need all three kinds of information, and in roughly equal amounts, for a solution to emerge. That’s fine for most Agile teams — we can get a lot through conversations and notes on story cards. The problem is that if you’re not completely sold on Agile, you start focusing on those User Stories, on the backlog, and start thinking that the harder you work at the backlog, the better the future product will be. That’s simply not the case. In fact, it works the opposite way. You end up with a perfect set of specifications and designs for a system that takes way too long to build.

A practical, short book on Agile Backlogs

All of this is why our second book in our Tiny Giant series is on backlogs. It’s the thing we’re most unhappy with. Most teams have backlogs that need quite a bit of work — and they know it. It’s the thing that’s most-misunderstood about actually implementing Agile. A simple-to-understand and short book on Agile backlog management, grooming the backlog and agile backlog formats has to be something that can bring value to the community.

The idea that anything that humans can do can be broken down and managed scientifically is true when it is limited to a narrow set of activities around producing concrete items. But when the product becomes more like intellectual property, the process boundaries are no longer concrete — ideas don’t move around your team on a conveyor belt — and the paradigm fails. And even when it does work, nobody wants to be treated like a robot. Do yourself and your team a favor: take some time today and think hard about the state of your agile backlogs.

Follow the author on Twitter

January 25, 2012

7 responses to Agile Backlogs. Sigh.

  1. Maris Berzins said:

    Big backlogs are a waste of everyone’s time. At my last company, we wrestled with it for a long time, with our use of SCRUM. We later switched to Kanban, which forced us to change our thinking around backlog management — you only “managed” the next set of stories to be considered. You ignore the rest. It was eye opening. If you are not familiar with Kanban, you need to check out Anderson’s book. [Great book!]

    • Paulus Meijs said:

      I wonder whether the difference is made by the use of Kanban instead of Scrum. Maybe in Kanban you are even more focused on “the next item ONLY”, because that is the one that you are going to pull in the process. But still you have to choose which one is the next one in line. Mostly a “first in, first out” will not suffice, so that you need to balance a new item with the already existing, in order to determine the new order. I would think that you can run into the same problems as with a Scrum backlog. And then the scheme told above is a good approach.

  2. hipcheck said:

    This sounds nice, but I find that going too far toward Simple doesn’t work when you’re working in big programs. I have yet to manage a project that isn’t part of some program that can’t go completely Agile (for ex they have hardware that must be built before being assembled before being tested). Try convincing a program manager in such a program that you’re only planning one Sprint ahead, or hardly managing your backlog… it’s enough of a stretch for most of them that you’re working on such ‘thin’ requirements to start with.

    • admin said:

      “…Try convincing a program manager in such a program that you’re only planning one Sprint ahead, or hardly managing your backlog…”

      Either you misunderstood or I didn’t explain well enough.

      A team should be able to reliably predict when any item on it’s backlog will be delivered into production.

      Backlogs should be managed to the degree necessary to support whatever environment the team is in. If you’re in a heavily contractual environment, for instance, you may have to do a lot more documentation. The purpose here is to identify the least amount possible, and at the last responsible moment. Not to have a sloppily-managed backlog.

      Also remember that things like the limits in sizes for lists people can work on is not negotiable. Even if you had a customer that wanted a 3 thousand item backlog (and I’ve seen backlogs as big as 8 thousand items), you’d still have to work in a format that the team could absorb. The rest would be just noise — people can process only so much stuff. That limit has nothing to do with Agile. (And explains why many well-meaning documents, briefings, and meetings do not accomplish what they set out to do.)

      Thanks for the feedback. I’ll try to do better explaining that in the future.

      • hipcheck said:

        My comment stems from pushing Agile in very non-Agile environments for the past few years. I had one (non-Agile) list of requirements of 40k items for one product… very true that this shouldn’t be a headache for anyone who is not a PM or Req’s mgr.

        These days I tend to mix kanban with scrum to the effect of having not much more than the current sprint in the minds of the dev team… and like Agile philosophy suggests, we keep things lean on the paper and docs end, and let the product people worry about writing tons of docs.

        • admin said:

          I’ve worked in both large organizations with PMOs and detailed product structures and “traditional” small web programming Agile shops. I’ve had these discussions and fought these battles many, many times before!

          On one hand, the “pure” Agile people will tell you that _everything_ is emergent. When asked to scale, their answer is just a bigger version of the team idea. Got five thousand people? Then you’ll need a scrum-of-scrum-of-scrum-of-scrum-of-scrums.

          I find this to be unsatisfactory — and counterproductive. It simply doesn’t work that way. As you pointed out, if ANY part of the organization does things the old way, you’re going to have friction. And compromise.

          On the other hand is folks who think (and use) Agile as just another means to micro-manage teams. To them _nothing_ has changed. Agile is just putting a new terms around stuff we’re already doing.

          I find this also unsatisfactory. There’s real potential for huge change in Agile. It’s not a magic bullet, but by comparing notes and learning how to learn, learning how to use Agile principles to make the larger organization itself more Agile, we get much more competitive and efficient. I honestly believe it can be transformative in a way that other styles and fads never were. Google is running 40K developers in a very Agile way. All sorts of companies, including industries that have little to do with computers, are getting into Agile. That’s not because it’s just fashionable. There’s really something there worth getting. Especially when you start talking lean startup principles.

          When people on both sides of a discussion are making broad claims and speaking in absolutes, it’s tough to pick out the good stuff from the bad stuff, the applicable pieces from the dreaming.

  3. Pingback: What's a Backlog, and Why You Should Care - Bret Wortman

Leave a Reply

Your email address will not be published. Required fields are marked *

* Copy this password:

* Type or paste password here:

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>