Category Archives: Python

More Expensive and Complicated Equals Better: Carseats and Software

So I finally got around to checking out the TED site; I’ve quickly become a fan. One of the first talks I watched was Steven Levitt’s child carseats talk. Both the talk and the feedback comments on the TED site reminded me of things I see in software development. Here’s the video if you haven’t seen it.

Steven Levitt shows in his talk how 30 years of data on car crash fatalities imply that carseats do not outperform regular seatbelts for children ages 2 and up. Anyone who has a child or grandchildren will probably bristle at hearing that; as the father of a pre-schooler, it certainly gave me pause. We spent no small amount of time and decent chunk of money in selecting carseats for our child, thinking we had done our best to ensure our child’s safety. For that matter, it would be illegal for us not to have done so. To see a decent-sized data sample suggesting my child would be better off in a seatbelt at her age is rather unsettling.

By the end of the talk, I took away these observations:

  • This great swath of an initiative was started by a very small yet vocal group of proponents whose assertions received little scrutiny.
  • A solution tailored to the specific constraints of a portion of the audience has been put forth as the de facto solution for the entire audience.
  • There is an ongoing assumption that the more complicated and costly solution must be superior to the simpler, existing solution.
  • Cursory evaluation of a data set without rigorous attention to mitigating factors can lead to wildly inaccurate conclusions.
  • The acceptability of a given solution is often tested against a narrow band of the overall criteria, and the solution is often optimized to pass that test.
  • The likelihood that people will continue to choose the more expensive and complicated solution despite any data is high.

So, software professionals, does any of that sound familiar? It reminds me of numerous initiatives over the years that have led us inexorably to the software productivity morass we have slogged through for years now. I suppose the most heinous case study in my own experience would be J2EE and the insistence that it was the solution that any reasonable business application would choose for a platform. (Before you .Net folks jump on that one, DCOM and later the .Net enterprise stack was much the same.) And who hasn’t read a benchmark or white paper with seemingly incontrovertible data depicted in highly-polished graphics insisting that Product Y is the one solution to address them all?

I noticed that there were comments below the video on its page at the TED site, largely because the negative verbiage of the topmost comment jumped out at me. I took the time to read through them all (there were 36 at the time of this writing), and to my amusement, I saw parallels between them and how people react to questioning and examining our existing practices and means of developing software. See if any of these strike a chord:

  • The idea that what we’re doing might be wrong unsettles some people, and their response is often ferocious and irrational defense of the status quo.
  • Those who have a financial stake in the current mode of operation are less likely to be open to scrutinizing it.
  • There will be some people whose primary objection to the scrutiny is that they didn’t think of doing it first; these people will sometimes attack the person questioning things on the grounds that it’s all a selfish, attention-mongering endeavor.
  • There are some people who will also be open to examining why and how we’re doing things; whether they conclude the same thing or not, they are a welcome presence among the larger mass of those who ardently refuse to entertain the possibility of a need to change.

In recent years I’ve been greatly encouraged by the willingness of companies to question whether or not the heavyweight frameworks and technology stacks are what they should be using. Helping companies slough off high-ceremony processes in favor of right-sized process that focuses on delivering the right software in a timely manner has been some of the more rewarding work I’ve done the past several years. I think we have an encouraging number of people in the industry who are challenging the “more expensive and complicated always equals better” mantra.

For the brave individuals willing to put these questions to the community at large, I hope you find some comfort in knowing that the resistance and rejection you will encounter is a thing to be expected; you are not alone in that respect. Here’s hoping we continue to be open to self-examination, no matter what emotional responses it might provoke or what fear of the future it may stir up within us.

Keep a long-term perspective; know your “sales cycle”

(This is part two in a series I have titled “Soft Skills in Software”, which came out of points I came up with for a panel discussion at CodeMash 2008.)

When introducing change in a technology practice, you can save yourself some frustration by being aware of the “sales cycle” for new ideas within your group and maintaining a perspective that always bears the long-term in mind.

As technologists, we often have great enthusiasm for new ideas that we perceive to hold value in terms of their (potential) positive effects on our work. The value is so apparent to us that we often see no reason not to put this new idea to work right away. However, the speed with which a new idea can be adopted is affected by a number of factors. Let’s start with these:

  • The organization’s general disposition toward change
  • The change agent’s standing as a thought leader within the technology practice
  • The resultant impact of adopting the idea in question
  • The temporal context in which the new idea is introduced

Depending up on the value of each of those variables (and more), the length of time it takes for a change to be adopted can be anywhere from a few weeks to months to years — or a special category I have, “not for the foreseeable future”. I use that category instead of “never”, because there have been times when I thought there was no way a given thing would ever happen, and suddenly a merger or regime change or some other major external change suddenly opened the door for the previously-infeasible.

I use the term “sales cycle” because it was in a previous career of mine where I first learned this lesson. I used to sell industrial packaging and marking systems, and when I first started, I was greatly frustrated at the seemingly low number of sales that I would close. I would follow up with some clients to the point of irritating them, convinced that “something should have happened by now”. I eventually learned that the different types of systems that I sold had different sales cycles. Because of a given system’s cost and the role it played in the manufacturing process at large, there was a minimum amount of time it took to get approval, move through a cusotmer’s procurement process, and finally issue me a purchase order. Some systems took 4-6 weeks to sell, while others were 6, 12, and even 24 months. In general, the longer the sales cycle, the bigger the sale. Once I understood this, I found myself far less frustrated, and there were fewer of those awkward interactions between my customers and I.

Introducing new ideas to your development group is quite similar. There is a direct relationship between the level of change and the time it takes for a team to agree and then adopt the idea. I have witnessed more than a few programmers who had good ideas that they presented and expected everyone to just approve right away. Their frustration with what they perceived as stalling and/or rejection typically resulted in undesireable outcomes. Even the most-valued technical person is only indulged a few temper tantrums before being labeled as “damaged goods”. Recovery from this sort of thing within an organization is discouragingly rare. The person is usually avoided in discussions about strategic direction, and their ideas are usually dismissed or stifled. In general, if letting person Y talk about changes has resulted in discomfort in the past, the typical solution is to just not let that person talk about changes. It’s warped, dysfunctional, and counterproductive, but that’s mainstream corporate culture for you.

One area where technical change differs from sales is the possibility of incremental change. It’s not always possible to sell only a piece of a large system; changes in software development, on the other hand, are well-suited to incremental change. Being able to adjust one’s expectation from demanding a single big win to being content with a series of methodical, steady, small victories is a valuable skill in software. For some big changes, that can be the only way to introduce them. For example, a company may not be open to switching to Agile Software Development overnight, but having daily stand-up meetings and developing in 2-week iterations certainly seems benign enough.

Having said all that, there have been times where something with a 12 or 24-month sales cycle was adopted immediately for me. Urgency, crisis, severe failure with the current situation, any number of things can result in exceptions to the rule. The art is knowing when to spot these factors and acting appropriately to leverage them. If I could sum all that up in a blog post, I’d probably be in a different line of work ;-).

Mark Ramm, friend and exemplar

My post about a Python web frameworks OpenSpace convened at CodeMash drew a bit of attention, and looking over it again, I can see how it can send messages I did not intend. In order to break my trend of sitting on posts for weeks and months, I pushed myself to get this one out even though I wanted to finesse it more. The main person who could have taken this the wrong way would be Mark Ramm, who convened the talk and is also currently heading up my favorite Python web framework.

Instead of launching some missive or flaming the comments of my post, Mark took the time to further explain the situation for TurboGears both past and future. He even apologized if he may have unintentionally spoken ill of the JVM (which he did not). Mind you, I am no defender of the JVM, but I can see how I might seem like it from that post. Mark also sent an email which explained some of the discussion items and after hearing that, I was far less concerned about where things are heading with that project.

I have known Mark for about a year now. We have had dinner together and shared many funny stories (he has many, should you ever have occasion to hear them). I consider him a friend, and his reaction to that post is an example of what I have come to enjoy about the Python community. After reading Zed Shaw‘s “Rails is a Ghetto” tonight, I am even more thankful for it. Man, even if you filter through the bile and profanity, it still sounds like this dude received some rough treatment for trying to do the right thing at times. And if some of it is true, then I need to rethink some of the folks whose work I endorse.

Curb Your (Python Web Framework) Enthusiasm

One OpenSpace topic convened at CodeMash was “Python Web Frameworks”. TurboGears seemed to be the most-represented citizen of the Python web world, but some asserted that they use Django. I had skipped Brian Goetz‘s “Java Performance Myths” talk in favor of this, even though I am sure that would have given me more ammo to smack down the crusty turds in the Java world who are always denigrating the work of others for its lack of their arcane incantations that are allegedly “critical to performance”. I have to say I was pretty dismayed after attending the OpenSpace, but maybe that’s not a loss after all. Perhaps my Python web enthusiasm was due to be curbed. Also, maybe I shouldn’t let those crusty Java turds get to me so much, even if I do feel they are a key agent in keeping the Java community from being more inviting.

Coming away from the discussion, my impression is that the Python web framework community is perfectly content with the dilution of effort and momentum that is caused by the proliferation of web frameworks. We talked about the number of projects out there, laughed a little, and then folks seemed to be ready to move on. One participant fortunately asked the question “How are TurboGears and Django different, because they look the same to me?” That’s a paraphrase, but the gist of his whole question was, as the two leading Python frameworks, how are these two serving different needs, and if they’re not, why are there two? What followed was some dialog that choice was good, the usual stuff you’ll hear in Open Source discussions about duplication and fragmentation of effort.

I followed up on his question commenting that the degree of fragmentation has seriously hurt the credibility and perceived viability of Python web frameworks in larger shops that are entertaining dynamic web frameworks as possible alternatives to doing everything in Java or .Net. I added that the proliferation of choices in Python has really only served to dilute the attention and effort for those that are emerging as the leading ones. I also shared that those outside the Python community are perceiving its web offerings as momentary fascinations of the brilliant, yet quirky enthusiasts upon which nothing long-term should be based. This didn’t seem to be well-received by the group, but I may be wrong. I think most folks present may have been up for a Python love feast instead of some serious introspection on where we are and how we got here.

Another thing we had was Python fans without significant Java experience continually emitting FUD about the JVM, which blows my mind. I have been following Python pretty closely for over a year now, and the amount of ignorance about Java in the Python community still surprises me. The lack of hands-on experience with Java does not seem to deter the constant, half-formulated criticisms of it. Some even seemed to be implying that Python’s threading situation was better off than Java. The platform of Java quite frankly kicks the hell out of most all others in that arena; this is widely-known.

There is an unsettling meme that seems to be running through the Python web framework community of “doing what’s cool”, which is unsettling. Cool and pragmatic seldom keep close company. One of the most heartening things about Rails 2.0 has been the steps taken to pay down technical debt and bolster the soundness of the framework. Hearing that TurboGears is pursuing X because Django has it and that’s cool disturbs me. “Cooler than Y” doesn’t get me a framework I am comfortable building upon; robust and reasonably-documented does.

I viewed the merged effort with the Pylons project very positively when I learned of it. Collaboration and true sharing are all-too-absent in F/OSS these days. However, the more I learn about the culture and mantra of Pylons, the more I am inclined to think it may have been an ungainly partnership. I am wondering if TurboGears’ original “compose a mega-framework of best-of-breed” may have subtly morphed its perception of best-of-breed to be “coolest”. And cool, like beauty, is in the eye of the beholder.

The need for a degree of stability in a web framework is not unlike the dilemma faced by technical authors and producers. Something I have heard from multiple technical writers is that one must resist the temptation to be forever revising the work in hopes of keeping pace with the changes occurring in the technology being covered. In software that rate of evolution is such that it is a Sisyphean undertaking. One must pick a point of time and stick to it. The hope is that said point is an acceptable level of stability so that the book is not useless weeks after it hits the shelves. (This begs the question of whether or not many technical books in print today should not have been published, but that’s a separate topic.)

My chief concern for TurboGears is that the apparent direction toward “cool” is in direct conflict with the need for stability and documentation. The quality of Open Source projects these days is such that few will tolerate a poorly-documented project that poses an unnecessary learning curve primarily brought on by the need to figure things out for yourself. Unless the documentation is shored up and the rate of significant component change slows, I think TurboGears will be challenged to gain more traction in the wider web framework world.

The reason I am even bothering to write about this is that I care about the TurboGears project. I chose it for my framework to explore at Bruce‘s Dynamic Web Frameworks Jam, and was blown away by how much had been accomplished in so little time with the project. It had things I have been wanting for years in Java web development, and the learning curve was so low that I found myself running ahead of my own understanding of Python at times. The DRY-ness of TurboGears is another strong draw for me. Implementing your own components when superior, highly-tested ones exist is just silly these days, unless considerable mitigating factors are present. While I think it holds lots of promise, I can’t say that I am comfortable basing any significant projects on TurboGears right now.

CodeMash 2.0.0.8; why I choose to attend

So tomorrow I fly out to Cleveland for the 1-hour ride to Sandusky, Ohio for CodeMash. The first CodeMash ever took place last year, and I was very glad to have been a part of it. I had intended to submit proposed talks for this year, but alas my schedule was already a bit full and I honestly lacked the energy to put together the topics I would have wanted to present. I cannot wait to be there, several of my colleagues from Bruce Eckel’s OpenSpace conferences will be attending.

One benefit of being an independent consultant is that I am able to choose the conferences I attend without having to justify it to some management or department head. Most of the things I choose to attend would probably not be approved by an organization that insists on being able to directly relate the entirety of a conference’s content of focus to the attendee’s daily tasks. Still, there are numerous conferences throughout the year that appeal to me, and I can only afford to be unbillable for a certain number of weeks out of the year. CodeMash makes my shortlist, and here are a few reasons:

  • CodeMash is a multi-language conference. There’s a breadth of exposure and a cross-pollenation of ideas that are simply not available at single-language events.
  • The attendees drawn to such events are typically more open-minded about software development approaches and the fact that there is no “one way” or “one language” for anything.
  • The speakers drawn to this type of conference tend to have a perspective that is broader and more in touch with software development realities than insisting on foisting some delusional monocultural vision for their language/platform/tool of choice.
  • There’s a strong OpenSpace element to CodeMash, and I have experienced first-hand how amazing and energizing it can be.

So, that is why I am going to CodeMash. Looking forward to sharing my experience with my readers, all three of them.

Java and the prodigal son syndrome

Ever hear the parable of the prodigal son? If you have had any length of exposure to the Christian New Testament, the answer is probably yes. It’s this one parable Jesus shared, captured in the Gospel of Luke, chapter 15, verses 11-32. A depressingly large contingent of the Java&#8482 community shares some annoying similarities to that prodigal son.

The parable is actually about two sons, not just the one who ran off with his share of the inheritance and blew it. There’s also an older son, who was always a good worker and didn’t run off. When you hear someone speak about this parable they usually point out how he’s bitter that the family celebrates when the kid who blew half of everything finally returns home, and that you shouldn’t be that way, et cetera.

A friend of mine once pointed out to me this subtlety; only the father comes out to speak with the older son. The younger son, who has just received this abundantly merciful treatment by the father, makes no gesture to come out and reconcile with the older son. He has quickly forgotten what it was like to be the alienated one. My friend John called that the “prodigal son syndrome”; I think of it often when I listen to and/or read reactions of Java &#8482 programmers to…just about anything other than Java&#8482.

For those in the “Java™ will never cease to flourish” camp, recall some of the scoffing you experienced back in the early period of Java™’s adoption, the jeers from the C++, C, and maybe even COBOL camp, depending upon the nature of the shop you were in. Now, take those scoffs and perform a search-and-replace:

s/Java™/{Ruby,Rails,Python,etc.}/ && s/{C,C++,COBOL}/Java™/.

Listen to yourself when you respond to topics like Ruby on Rails. The resemblance is unsettling, don’t you think? Why must everything new (or non-Java, really; newness isn’t even a condition most of the time) be rejected out-of-hand as inferior and “a fad”? The unwillingness to acknowledge (much less embrace) the strengths of alternatives to Java&#8482 ironically serve to undermine your puerile assertion that Java&#8482 is the be-all, end-all in programming.

The ardent defensiveness in the Java™ community’s response to Bruce Tate’s Beyond Java™ awhile back is an example of what I am talking about. I’m too busy these days to catalog some of the shining examples from java.net and TheServerSide. But good grief, people, drop it already! And don’t be embittered when you’re overrun by the changes you continually dismiss as inferior to Java &#8482. There’s an open cubicle down by that grumpy COBOL guy; maybe you two can commiserate.

NOTE: Apologies in advance to the sed crowd for my cruddy syntax above.

So what is a server-side guy doing at the Java Posse Roundup 2007?

The Java Posse Roundup 2007 in Crested Butte, Colorado has the central theme of “Java off the server”. Given that topic, why (besides the skiing) would I haul out to Crested Butte for this meeting? In short, I am here because I know what Open Space events organized by Bruce Eckel are like and I know there’ll be plenty of topics that are not strictly “off the server”. The first day already made good on that prospect. Heck, Python and Domain-Driven Design came up within the first topic I attended.

The initial conversation I participated in was related to the role of dynamic languages on the Java Virtual Machine (JVM) and their integration with Java even in their native implementations. Most folks seemed to agree that the JVM holds much promise as a platform with mature threading and stability for dynamic languages whose C implementations suffered in those areas.

As an Open Space veteran, a meta-topic for me is watching folks who seem to be unsure about whether or not we’ll get anything done with this meeting format. I look forward to hearing their impressions at the end.

Skiing this week is excellent! Good snow, not too crowded, good weather; this will be a great time.

CodeMash – a rich, inspiring experience; more to come

So, for the latter half of this week I have been at CodeMash in Sandusky, Ohio. It has been one of the best conferences with traditional-style presentations that I have attended for some time. We have had some remarkable Open Space sessions interspersed throughout the regular agenda. I think most of us are saving up our posts for afterward; it’s been a pretty non-stop pace. The last group I was talking with yesterday wrapped up after 1:00 AM, if that gives you any idea. This has had that same cross-platform, cross-disciplinary feel we had last March at Bruce Eckel’s Programming The New Web. Lots and lots of great stuff. I only hope I can capture a good bit of it and share.

Web Frameworks Jam Wrap-Up

The last day consisted of a steady morning of TurboGears activity followed by lunch, a great hike, and wrapping up with a podcast that summarized our experiences in the different areas. I’ll add a link as that becomes available. I didn’t get into the testing capabilities bundled with TurboGears to the degree I had wanted, so I’ll have to do that later and blog about it.

We also rented bikes for hitting the Lower Loop at 8:00 a.m. tomorrow; I eneded up with a Gary Fisher Piranha from The Alpineer. It’s not quite my Hoo Koo E Koo, but at least it has the same Genesis geometry; that Specialized bike I rode on Monday was a dog, though that was mostly due to it being horribly unmaintained. The geometry on it was really awkward for me as well.