Tag Archives: Lean

Empirical Process Control: Why Scrum Works

Being in my ninth year of applying the processes and technical practices of Agile and Lean software development, people are sometimes surprised to hear me say that Scrum is still my preferred process. Let me explain.

Scrum was designed to enable empirical process control for software development. Of all the things that are important to understand about Scrum in particular and Agile/Lean concepts at large, this is the one fewest people seem to have heard. Why is this important?

Scrum’s foundation in empirical process control is important because its model that fits the realities of creating software. Consider the following succinct definition from Wikipedia’s short entry on the topic:

The empirical model of process control provides and exercises control through frequent inspection and adaptation for processes that are imperfectly defined and generate unpredictable and unrepeatable outputs. – Wikipedia – Empirical process (process control model)

Let that compound sentence sink in for a moment. How effectively does that summarize your experience as a professional involved in the development and delivery of software?

Using empirical process control requires three basic elements: transparency, inspection, and adaptation. Transparency ensures that all the elements a process (I often define that as everything involved in going from concept to customer) is openly observable. Inspection is the activity of taking that observation enabled by transparency and critically evaluating how work flows through the process (in software, our cross-functional team). Adaptation takes the insights gleaned from that inspection as a basis for making incremental ongoing improvements to the process.

Scrum’s transparency comes in the form of an openly viewable Product Backlog, highly-visible information emitters in the form of Task Boards and Burndown Charts, a Daily Standup, Sprint Reviews, Retrospectives – all of that exists to clearly convey the flow of work through a cross-functional team. Scrum’s inspection comes in the ongoing daily observations and interactions of the team in moving work across the Task Board, and culminates in the Retrospective with the open, healthy discussion of what is working well and what is not working well. Scrum’s adaptation begins in the Retrospective, where the team identifies a few things to change in the next Sprint, then continues forward as those changes are implemented.

There is an indirect benefit to this application of empirical process control that, in my estimation, outweighs its immediate value. Through this process, people across functional designations are interacting with one another more than they ever had before. As they interact, they not only do the work, they think and dialog about how they do the work. Built into the cycle of transparency, inspection, and adaptation is this ongoing mental prompt for the group to ask itself, “Now why is it we are doing things this way?” Year after year in many different settings I have seen this result in a healthier group of people who are steadily improving how they work together.

Ken Schwaber emphasized the centrality of empirical process control quite a bit in the book Agile Software Development with Scrum, but it’s in the second half of the book. People rarely get all the way through books these days. So, I’m not exactly shocked when I bring up empirical process control when talking about Scrum and Agile and people tell me it’s the first they’ve heard about it.

Do not take these words to imply that Scrum alone is sufficient in creating a healthy Agile/Lean team. Mind you, it’s always accompanied by the other things that make for a holistic Agile/Lean environment: User Stories, testing (preferably Test-Driven Development, but not to the point of religious zeal), effective use of version control systems for change isolation and integration via branching, merging, etc., continuous integration, and, most importantly, a culture that is healthy enough to give all these things a fighting chance at taking root.

Barry Hawkins of All Things Computed coaches groups to successfully apply the processes and technical disciplines of Agile/Lean Software Development. In addition to coaching, Barry practices what he preaches when he develops software on contract as work for hire.

Is Agile/Lean as good as it gets?

I was recently catching up on episodes of The Java Posse, and came to the one that features an Open Spaces topic I convened at last year’s Java Posse Roundup, Should We Shoot Agile in the Head? It was a vibrant conversation with plenty of input from experienced people. At the 49:50 in the recording, D. J. Hagberg posed an excellent question:

“Is Agile[/Lean] as good as it gets?”

I added Lean, since the question really applies to both, their overlap being so great.

The short answer is no.

What D. J. is asking in the context of that conversation is something I wish more Lean and Agile practitioners were asking themselves. Is the full-fledged adoption of a given process as canonically defined by some authority the goal? If so, to what end? What does that buy us? Is that a guarantee of success?

I will tell you what is “as good as it gets”.

Working as part of a group of talented, disciplined, and motivated people to build excellent products in an environment where trust and collaboration cross the boundaries of functional roles with just enough structure to hold things together is as good as it gets.

Agile/Lean processes, tools, and practices can greatly facilitate those things, but they will not create it. At the end of the day, yes, you are in fact going to have to get better at working with others. If you are applying anything Agile or Lean in your teams, ensure that it is with the aim of fostering trust and collaboration, executing tactically with strategic vision in mind. And yes, that is not easy; excellence has never been easy.