Free Clues from Eric

In mid-October, Eric S. Raymond's book The Cathedral and the Bazaar came out from O'Reilly & Associates. It contains updated versions not only of Eric's landmark title essay, but also both companion pieces, “Homesteading the Noösphere” and “The Magic Cauldron”.

We thought it would be a good idea to interview Eric about the new material in his book—and, since this historian of the hacker tribe never stops thinking, to glean a few of the ideas which have evolved out of his brain since finishing it. This conversation took place on September 22, 1999.

Doc: So now that we have “The Magic Cauldron” in final form, what new ideas are you working on?

Eric: A couple. I'm further developing the critique of conventional software-development management that I outline in the new last section of “The Cathedral and the Bazaar”. Also, I'm noticing some interesting developments in the business ecology of Linux.

Doc: Let's take the development management idea first.

Eric: I've been developing my thoughts about play being the most efficient form of work. Let's start by asking a basic question: What are the circumstances that make a programmer productive?

A programmer is productive when he is neither under-challenged nor over-challenged—neither bored and underutilized at one end of the scale, nor burdened with artificial deadlines and technical and procedural hassles at the other end. When the programmer's skill and energy are well-matched to the task, that's when you'll see the maximum productivity. The interesting insight is that these are also exactly the circumstances under which the programmer will be happiest, when his enjoyment in writing good code is maximized.

So in order to get the maximum economic leverage out of the guys you hire, you need to reduce friction costs at the top end, because you'd rather pay for their code than their grief. On the other hand, you don't want them to be bored—because when they're bored, you're not extracting maximum value from their ability.

The general prediction is this: enjoyment predicts efficiency—and not just in some fuzzy New-Agey way, but in a hard-nosed economic sense.

If you use your developers the most efficient way you can in creative work, they will be happy. On the other hand, if you have a development management style or an institutional tradition under which your developers are bored, frustrated and angry all the time, you are failing. And not just humanistically, but economically. You are failing to extract maximum productive value out of your people.

Doc: You have Dilbert's work environment.

Eric: Right. If you look at traditional software development management, you'll find that of the five classic tasks of the manager, maybe four just don't apply at all in an Internet environment. It's not so much that the traditional assumptions are wrong in some deep sense, but they're irrelevant: they proceed from assumptions that no longer match the problem. Because the assumptions don't match the problem, we've got an impedance mismatch: a lot of bored, frustrated, angry and unproductive programmers out there.

Now there is a popular “burger-flipper” school of management that says this kind of unhappiness in production workers doesn't matter as long as they're still cranking out a minimally acceptable product on the right deadlines. In fact, some managers just sort of gut-level assume that people aren't working hard enough unless they're stressed out, so they figure the right thing to do is apply the whip every once in a while just on general principles. This is the thinking behind artificial deadlines—it's the Simon Legree theory of motivation.

But a funny thing happened on the way to the plantation—the overseers themselves are revolting. Managers are increasingly frustrated, because they're doing all the supposedly “right” things, and their jobs and their results still suck. The days that only engineers griped about this stuff are long past. Today, Dilbert cartoons hang in executives' cubicles.

Think about that. If the system was working, would the managers display contempt for it?

Doc: Tom Peters has observed that the best-selling business books are his and Scott Adams'. And he laments that all those Dilbert books are relentlessly cynical.

Eric: I'm totally with Peters on that one. When I speak to CEOs and investment bankers, I try to wake them up to the fact that the popularity of the Adams books all the way up the management chain is actually not funny. It means the system is broken. If we're going to fix it, if we're going to get our productivity to where it ought to be, we've got to start thinking humanistically about what makes people happy at work. We've got to banish the Dilbert syndrome.

Doc: What about this ecology thing?

Eric: What I'm noticing lately is that Red Hat has developed a couple of satellite distributions that are significant in their own right. One of these is Kevin Fenzi's Red Hat Über Distribution (KRUD)—which I am now running on my own machine, by the way. The other is Mandrake.

Doc: These are cases where others are leveraging Red Hat.

Eric: Right. Both looked at the Red Hat distribution and said, “I see a significant value-add here that Red Hat is either not interested in or not in a position to exploit.”

In the case of KRUD, Red Hat doesn't put out updates often enough. So what Kevin does is press CD-ROMs every month on a subscription basis. A new one shows up in the mail, you stick it in your drive, go through the update procedure that pulls all the new RPMs off the CD-ROM and installs them, and you are completely up to date.

I think of this as the magazine model of operating system publishing. Want all the latest updates? Subscribe—it's just $36 a year. In return, you don't have to spend a lot of time downloading stuff and worrying if you're current.

Doc: Who's the customer?

Eric: KRUD is particularly valuable if you are a site with serious security requirements. Want to be sure you have all the latest security patches? That's what Kevin does for you. He's the maintainer of the Linux Security HOWTO. He's at http://www.tummy.com/, by the way.

Doc: Why can't Red Hat do this?

Eric: Scale, I think. They have to ship CD-ROMs in sufficiently large runs to carry their other expenses. They would be bleeding all over the place in fulfillment and marketing expenses if they tried to compete in this new space by going to a one-month rather than a six-month release cycle.

Doc: Red Hat only scales higher up the value chain.

Eric: Right. Kevin can do this because it's a relatively small effort for him, and he doesn't carry all of Red Hat's development and marketing overhead.

Doc: What about Mandrake?

Eric: Their value-add is that most people have Pentiums now, and Mandrake can take all the source RPMs and recompile them with Pentium optimization so your machine runs 30% faster. Again, it would be too expensive for Red Hat to have two distributions, one optimized and one not.

Doc: Does Red Hat benefit here?

Eric: Sure. Red Hat is certainly growing their support market this way. If I'm a Fortune 500 site and I have a subscription to KRUD and have a support requirement, do I go back to Kevin? No, because I know he's just one guy sitting in a room in Colorado somewhere. So I go back and buy a support contract from Red Hat.

Doc: And if you're in the same position with Mandrake?

Eric: You may go to LinuxCare. I believe they're Mandrake's partner for service. So the Mandrake/LinuxCare combination is competing with Red Hat a little more directly for service revenue. Still, the Mandrake guys have a good working relationship with the Red Hat guys. In effect, Mandrake is acting as an R&D arm for Red Hat, because the changes they make, to fix and clean up things, can get folded back into future Red Hat distributions.

Doc: And the ecology point?

Eric: Because everybody is playing by the open-source rules rather than competition, there's a potential for cooperation to make more money. There is a potential for single distributions to speciate into mini-ecologies of mutually supporting distribution variants. These guys all share core DNA, and that's what makes the ecology work.

Doc: I imagine there is some analog in original markets.

Eric: Oh, sure, it's a value network in Christensen's sense. [Clayton Christensen is the author of The Innovator's Dilemma, an award-winning book on disruptive change in technology markets—Editor]

Doc: Have you got any more books in the pipeline?

Eric: Yes. I'm about halfway through writing the first draft of The Art of UNIX Programming, a book on how to think like a UNIX guru. It's aimed at a lot of the younger Linux programmers who have picked up bits and pieces of the UNIX design tradition, but don't really have the whole Zen. This is not their fault—because like Zen, it's only been passed along from master to student up to now—“a special transmission, outside the scriptures”. It's time to write it down.

Doc: Any other significant developments for you?

Eric: I'm doing most of my programming in Python these days. I've moved away from C.

Doc: Python is hot. It's taking our own people by storm.

Eric: Under present economic conditions, using a language in which manual memory management is required almost never makes sense. The thing about C is that you have to manage your buffers and dynamic storage by yourself. You don't have a garbage-collecting interpreter doing it for you. It's a huge source of problems—classically, something like nine out of ten of the runtime errors in C programs are memory-management screwups.

It used to be, back when machines were more expensive and programmer time less so, that it was worth hand-tuning resource utilization and putting in all the debugging time necessary to compensate for the high error rate. But unless you're doing systems programming or really heavy-duty number crunching, that tradeoff just doesn't make sense any more—not with cycles and memory as cheap as they are now.

Doc: Is that Python's main virtue for you?

Eric: One of several, but in my opinion it is the single most important one.

Doc: But other languages do the same thing.

Eric: Yes, there are lots of interpretive languages that do garbage collection, but very few are as well-designed as Python. The alternatives don't scale well; they make it harder to read and modify large volumes of code. I used to write a lot of Perl, until it got too painful. And the less said about Tcl, the better.

Doc: Closing thoughts?

Eric: Just a sound-bite version of what's beneath most of the open-source business models we're seeing out there. “Linux is free. Clues are not.”