Category Archives: Culture

IBM developerWorks Podcast: Barry Hawkins on agile software development

Earlier this year, my friend Andy Glover interviewed me for season 4 of his IBM developerWorks Java technical series Podcast (RSS, iTunes, MP3 download) on the topic of Agile software development. The episode is titled “Barry Hawkins on agile software development” and runs 41:03 in length.

This was an enjoyable interview; I’ve known Andy for years, and his style as an interviewer is quite comfortable and flows conversationally. What I really enjoyed about the venue is that it afforded me the chance to talk about software development, the Agile ecosystem, and the primacy of a healthy company culture in ways that I typically do in one-on-one or small group discussions, or talks at conferences that don’t get recorded.

Here is a rough outline of the discussion with timing marks should anyone wish to jump straight to a given topic:

  • 00:55 – How I got here: my journey into software and then Agile
  • 05:50 – Things weren’t always waterfall; Agile is a rediscovery of things lost
  • 11:12 – It turns out Agile is hard, but not because the old way was better
  • 12:25 – The underlying cultural elements of a healthy Agile environment
  • 20:35 – None of the Agile elements stand well on their own; they work best as complements to one another
  • 26:05 – As a newcomer to Agile, Lean, XP, and all this stuff, where should I start?
  • 29:25 – No matter what: Get better at interacting with the other groups in your organization
  • 31:30 – How to keep cross-functional interaction from becoming death by meetings
  • 36:35 – The scarcity of good ideas, and why they get expressed over and over

I hope you enjoy it as much as I enjoyed recording it. A big thank you to Andy and the IBM developerWorks folks.

Empirical Process Control: Why Scrum Works

Being in my ninth year of applying the processes and technical practices of Agile and Lean software development, people are sometimes surprised to hear me say that Scrum is still my preferred process. Let me explain.

Scrum was designed to enable empirical process control for software development. Of all the things that are important to understand about Scrum in particular and Agile/Lean concepts at large, this is the one fewest people seem to have heard. Why is this important?

Scrum’s foundation in empirical process control is important because its model that fits the realities of creating software. Consider the following succinct definition from Wikipedia’s short entry on the topic:

The empirical model of process control provides and exercises control through frequent inspection and adaptation for processes that are imperfectly defined and generate unpredictable and unrepeatable outputs. – Wikipedia – Empirical process (process control model)

Let that compound sentence sink in for a moment. How effectively does that summarize your experience as a professional involved in the development and delivery of software?

Using empirical process control requires three basic elements: transparency, inspection, and adaptation. Transparency ensures that all the elements a process (I often define that as everything involved in going from concept to customer) is openly observable. Inspection is the activity of taking that observation enabled by transparency and critically evaluating how work flows through the process (in software, our cross-functional team). Adaptation takes the insights gleaned from that inspection as a basis for making incremental ongoing improvements to the process.

Scrum’s transparency comes in the form of an openly viewable Product Backlog, highly-visible information emitters in the form of Task Boards and Burndown Charts, a Daily Standup, Sprint Reviews, Retrospectives – all of that exists to clearly convey the flow of work through a cross-functional team. Scrum’s inspection comes in the ongoing daily observations and interactions of the team in moving work across the Task Board, and culminates in the Retrospective with the open, healthy discussion of what is working well and what is not working well. Scrum’s adaptation begins in the Retrospective, where the team identifies a few things to change in the next Sprint, then continues forward as those changes are implemented.

There is an indirect benefit to this application of empirical process control that, in my estimation, outweighs its immediate value. Through this process, people across functional designations are interacting with one another more than they ever had before. As they interact, they not only do the work, they think and dialog about how they do the work. Built into the cycle of transparency, inspection, and adaptation is this ongoing mental prompt for the group to ask itself, “Now why is it we are doing things this way?” Year after year in many different settings I have seen this result in a healthier group of people who are steadily improving how they work together.

Ken Schwaber emphasized the centrality of empirical process control quite a bit in the book Agile Software Development with Scrum, but it’s in the second half of the book. People rarely get all the way through books these days. So, I’m not exactly shocked when I bring up empirical process control when talking about Scrum and Agile and people tell me it’s the first they’ve heard about it.

Do not take these words to imply that Scrum alone is sufficient in creating a healthy Agile/Lean team. Mind you, it’s always accompanied by the other things that make for a holistic Agile/Lean environment: User Stories, testing (preferably Test-Driven Development, but not to the point of religious zeal), effective use of version control systems for change isolation and integration via branching, merging, etc., continuous integration, and, most importantly, a culture that is healthy enough to give all these things a fighting chance at taking root.

Barry Hawkins of All Things Computed coaches groups to successfully apply the processes and technical disciplines of Agile/Lean Software Development. In addition to coaching, Barry practices what he preaches when he develops software on contract as work for hire.

The Discipline Deficit

I consult with all sorts of companies looking to adopt elements of process and practice from the Agile/Lean offerings. Whether it’s Scrum, Test-Driven Development, User Stories, Test Automation, Continuous Integration, Agile Estimation and Planning, Sprint Planning, or Sprint Retrospective facilitation, one challenge pervades across most scenarios. I have come to call it The Discipline Deficit.

A disheartening number of groups who take up a given Lean or Agile practice or process underestimate the amount of discipline required to master and sustain it. As a result, they either put forth a half-hearted effort and the attempt flops, or they get off to a good start only to abandon it. The unfortunate outcome of either of these scenarios is that the people who interact with said group are left with a rather bitter taste in their mouth, and as a result even the mention of the word Lean/Agile/Scrum/etc. becomes a trigger for painful memories and a general sense that whomever uses these terms is someone to be avoided.

I suppose this should not be a surprise; humans have a fairly poor track record when it comes to resisting promises of gaining something for nothing. There are more than enough opportunists out there selling Agile/Lean as a silver bullet of sorts that turns even the muddiest sow’s ear into a gorgeous silk purse while requiring little to no effort beyond mimicking several steps and leaving the existing culture intact, no matter how dysfunctional it may be.

So, once again let me declare that Agile/Lean practices and processes are neither substitute nor remedy for hard work, but rather effective vehicles for structuring and managing hard work. They are also wonderful tools for thinking about the way a group works and its culture, providing information that can be used to improve it – once more through hard work. If you are going to give any of these things a try, consider the cost both in effort and cultural impact. There is no shame in deciding not to apply them; in some cases it actually reflects wisdom and maturity.

Is it just my company that has a hard time with Agile?

After an Agile adoption is underway and the feedback mechanisms of most Agile processes begin to function, one or more team members usually ask me something like the following:

Is it just my company that has a hard time with Agile?

No, it’s not just your company. However, don’t take comfort in that.

I shared something on Twitter that I think sums up why companies have a hard time with Agile:

Agile practices and processes serve motivated, self-directed teams who work hard. It does not create them. @barryhawkins

When most groups decide to start embracing any Agile process or practice, they usually begin by attempting one or more of the discretely observable activities. This is almost always undertaken with no understanding of how that one piece fits into the cohesive whole of the given process or what underlying cultural elements are assumed to be present in order for it to work as expected. This is how you end up with a team having a “daily standup” (even better if it’s called a scrum) where most folks sit and talk about their day while staring at the wall, floor, or someone’s pen, waiting for the magic to happen. (Spoiler: It doesn’t.)

Why would a group even think this would work? Most likely because some overzealous (and typically unseasoned) practitioner or charlatan “coach” sold them on “Agile” (the fact that it was used as a noun is often your first red flag) as yet another silver bullet. You simply begin mimicking these few things that someone heard about in a talk (no need to read up on them; it sounded really simple), and sooner or later the excellence will kick in. The rhetoric usually reads like an infomercial for commoditized excellence in software development.

If you have a hard-working, self-directed team, an Agile process will most likely add some structure to your operation and show you areas to keep improving. If your team exists within an unhealthy culture, that is where it will begin to encounter its greatest difficulties. Effective cross-functional collaboration does not come about as a result of declaring a group “Agile”. The team will have to really push their ability to communicate and forge relationships with cross-functional groups. Be ready to adapt as you find out who is willing to work with you in this way and who is not. You can start a decent-sized cultural upheaval before you know it; count the cost of such a thing and decide if it’s worth it.

If you have an average team in a company where the technology is an afterthought, that process will probably be somewhat like salt on an open wound. Have you ever heard of a person who sent to rehab or for whom an intervention was staged, but they were nowhere near ready to admit they have a problem, much less deal with said problem? The outcome is seldom desirable, typically driving a wedge between those desiring to help and the one in need of help. Bringing an Agile process into a place that sees the current situation as perfectly acceptable is much like that.

If your team and the company it works for do not have a compatible culture for fully embracing a process like Scrum (that is my preferred one), prepare yourself mentally and emotionally for the fact that it will not “look like it does in the book”. That doesn’t mean you should not try. Almost any place there would be benefit from increased communication and accountability.

Agile processes and practices are hard for a reason; they are vehicles for pushing forward and honing your craft. Activities that drive toward mastery are always difficult; do not be surprised by this.


Ask an Agile Coach: What is an Agile Coach?

This installment of the Ask an Agile Coach series is a question normally asked by persons outside my field, but lately I have been asking it myself:

What is an Agile Coach?

Good question. I am not sure anymore.

The more I encounter others who call themselves Agile Coaches, the more I think I may not be up-to-date on what that is supposed to mean.

As I have historically interpreted it, an Agile Coach is someone who first actually established a proven track record of successfully leading a series of Agile adoptions as part of the actual teams. The insight, perspective, and expertise gained through that effort are what enable them to coach others. Multiple Agile teams in varied settings challenges what a person “knows to be true” and (hopefully) beats some humility into and the dogma out of them.

I have been under the impression that an Agile Coach also understands software development and the technical practices that support an Agile team. One must understand why testing, automation, code review, and pair programming exist. One must know what it feels like to work both with and without these things, know why professionals should adopt these practices, as well as when one or more of them are not a fit for the culture. It must be very strange to have management bring in a coach who tells you to do these things but cannot show you how.

I also thought that coaches should understand how to apply an Agile process in a way that stresses cross-functional participation. If what a coach is doing isn’t helping the company to function more effectively as a whole, I fail to see the point. Agile Software Development is neither a defense mechanism for engineers nor a tool for more effective exploitation by management.

It feels weird to have to point this out, but I believe a coach should understand Agile estimation and planning techniques, especially the ones laid out in Mike Cohn‘s books. Estimating and planning are among the most painful parts of software development, and one of the most fulfilling parts of coaching for me has been mentoring groups in Agile estimating and planning.

I also think an Agile Coach is someone who can say no. Agile Software Development is a challenge at best, and a cultural nuke at worst. There are places where the introduction of Agile process and practices would wreak havoc. I suppose the toughest part of this is that it requires a willingness to turn down paying work.

I am seeing things these days that make me think my idea of what an Agile Coach is might be incorrect. If it is, I might need to stop using that word. It doesn’t mean what I think it means.

Still a place for blogging

A few years ago, my blogging fell off sharply. There were multiple reasons for that. I began to spend my discretionary time elsewhere, including with my young child and playing World of Warcraft with my wife after the little one was asleep. Additionally, I began to lose interest in some of the main topics I had been writing about, in particular my involvement in the Debian project.

At that time, I began to wonder if perhaps blogging had become passé. Jokes about the ubiquity of banal blogs seemed to show up everywhere. I had adopted Twitter and Facebook fairly early, and I thought for some time that perhaps these social networking outlets were the successors to blogging.

They are not.

Blogging has always been a sort of mental sketchpad for me to tease out concepts, observations, and reflections that were too nascent at the time to make their way into a paper or a conference talk. I cannot think of any of those that would fit into a Facebook wall post, much less a 140-character tweet. I would find myself with an idea or thought in my head, but trying to cram it into these newer forms was so daunting I’d just let it pass. If I did try, it felt very constrained, like trying to give directions to your house in haiku or drinking a Wendy’s Frosty through a coffee stirrer. So, I have concluded that a blog still has a place for me, and some recent posts from other folks in my field lead me to believe that I am not alone in that viewpoint.

Any of you who used to read my stuff back when I was active will notice that I am no longer hosting this under the domain. My full name was not very prominent anywhere on the site, and now it’s in the URL. This was a change I had pondered before, but the advent of social networking made me certain I should go ahead and make the switch. Anonymity is increasingly scarce on the Internet, and since I’ve said more than enough out there to implicate me, I might as well embrace the trend.

So, welcome (back) and thanks for your time!