The Proper Image for Linux

Randolph Bentson

Issue #57, January 1999

Dr. Bentson did a survey of Linux kernel developers to find out about their backgrounds. Here are the results.

I get mail from folks about my book, the device driver I wrote for Linux, and about articles I've written for Linux Journal. A few months ago I got one which said, in part:

My boss is a great guy to work for ...[but he] is of the opinion that Linux is the work of “college punks” and will not consider it for serious work.

He had a nightmare with the MINIX file system and is permanently convinced that UNIX simply cannot be trusted and that Linux is the work of pimply-faced sophomores with time on their hands. I got a good laugh out of that while looking at your picture and reading your bio.

I can only hope his laughter was kindly. The opinions expressed by his boss weren't the first I've heard of that sort. Nor, I fear, will this be the end of it. Nonetheless, I decided to take a shot at confronting these claims.

I had suspicions that Linux contributors are a bright, experienced and well-educated bunch of folks. The discussions in the various Linux newsgroups and mailing lists aren't lightweight, nor is the resulting operating system. My “feel” of the operating system is that it's based on a lot of mature judgments and there is some theoretical grounding in what's being done.

Credits

I gathered up a list of contributors (from /usr/src/linux/CREDITS) and sent off 241 notes. Partial text of this note is shown in the sidebar, “Letter to Contributors”. I sent my notes with some trepidation—I didn't want to bother folks while they were working on important projects, and I feared a lack of response.

Letter to Contributors

I needn't have worried. So far I've received 103 replies, many of which have included a few words of encouragement. It seems that I wasn't the only one who wanted to respond to unjustified complaints about Linux. (Another 29 notes were returned with address errors. I hope to see corrections to the CREDITS file.)

Education

The level of response was the first piece of good news. The second was that I've been stunned by how strong the development team is with regards to both credentials and experience.

From these replies I found:

  • 1 had completed just basic public education (high school)

  • 15 had attended college or technical school

  • 23 had an undergraduate degree (B.S., B.A., etc.)

  • 19 had attended graduate school

  • 15 had a graduate degree (M.S., M.A., etc.)

  • 9 had done further graduate work

  • 19 had a terminal degree (Ph.D., M.D., etc.)

That's got to totally demolish the image of college hackers—at least the sophomore part of it. I figured I was an exception when I started working on the Cyclades driver while avoiding rewriting my dissertation. I thought, once folks were awarded a Ph.D., they would be busy with research, teaching or some other interest. I guess Linux development may be the doctor's favorite hobby.

When I offered an earlier summary of these results, my correspondent reported that his boss wisely intoned, “those folks are all academia and none of them have ever tried to run a business.”

Experience

I had sort of expected a comment along those lines and fortunately asked a few more questions in my survey. One hundred of the replies also reported the number of years spent programming or doing system design.

  • 4 had 1 year

  • 10 had 2-4 years

  • 31 had 5-9 years

  • 40 had 10-20 years

  • 16 had 20+ years

More than a few of us were programming before the integrated circuit came into general use. (Perhaps a mixed blessing—some of us may still suffer from post-FORTRAN syndrome.)

As I noted earlier, I have also felt that Linux has benefitted from a broad experience in its developer base. Linux may be a first operating system for a lucky few, but almost everyone (all but three) claimed to be at least a skilled user of another operating system. Eighty-three were skilled users of several other operating systems.

Nor was their contribution to the Linux kernel the first of that sort. Twenty have contributed to another operating system and another twenty-two have contributed to several other operating systems. One reported:

Speaking for myself, I had the same idea Linus did, but he beat me to it. (I've heard others say this as well.) I knew how to build a UNIX-like system from the ground up, and there was a need for it for PCs. (Vendors were charging exorbitant amounts for poor products in those days, and there was no good 32-bit development system for 386s.) I just didn't have the time. I had been playing with MINIX when Linus showed up on the MINIX newsgroups, and it took off from there. I can tell you that though I was a student at the time, I'd been a professional systems programmer for many years before. So, I and many others knew what professional quality software was, as well as how to produce it. I think it turned out pretty well.

Current Use

Finally, I wanted to know if the contributors were “doing Linux” in their careers. Eighty-two said their current employment was based on their computer skills. It was interesting to note that over a third reported their current employment supported or relied on their Linux development efforts. Sadly, two reported they were currently unemployed, but one of those also noted that he was “voluntarily unemployed to have time to put my life in better order.”

Perhaps one significant difference between Linux development and academic or commercial development is the duration of personal interest. In an academic setting, a student typically has one term, or at most one year, to work on any given program. When programmers leave a company, support is picked up by someone who has no sense of what has gone before. There is greater continuity in the Linux community because of the nature of submission and distribution. No matter what is happening at school or where one works day to day, contributors can keep in touch with progress on their piece of the puzzle. One person noted, “Personally, I did start my code in school, but that does not stop me from maintaining it now.”

Motivations

There are some other issues which weren't addressed by my survey. Although it might not seem relevant to quality and performance, a person's interest has a great deal to do with the outcome—it leads to a distinction between “craftsmanship” and “work product”. Another person noted:

“Intent” is what I think all of these debates are about. In the commercial world there is only one true answer to “Why are you helping develop Linux?”—“To make a living.” In the Linux community I'm quite certain the answer would be more closely aligned to “For me to use.” The Linux community tends to be self-driven and self-motivated, and that is what leads to the successes and the apparent failures in our development environment.

We are not a company; we don't have any one person, or group of people, setting the direction Linux will take. That direction is set by those with the energy to actually do something.

Another motive, akin to what pushed me to first join the effort, was shared by another respondent who said, “When I wrote [my code] for the Linux kernel I was working at [my former employer]. Linux use there was extensive, and I wanted to give something back.”

Motivation leads to the final and most significant issue—one which cannot be examined by a developer survey.

Quality

In a world driven by marketing, image is the basis for purchasing decisions. Even if a good image could be established for Linux by listing credentials or tabulating years of experience, I'd be reluctant to shift to that level. I'd much rather see acceptance and popularity for Linux based on quality and performance.

Even though I hadn't asked specific questions on this topic, a few people offered comments. One note seemed to identify, however obliquely, what may be the key to Linux's success.

In general, my experience is that most software I have seen which was developed by students is not of the professional quality I would like to see. On the other hand, much of the commercial software I have seen, which was developed by professional software development companies, is also not of the professional quality I would like to see. The difference is most people do not get to see the internals of commercial software.

Developing on this theme, another wrote:

The reason Linux is stable and usable is not because of its student programmers [or lack thereof]. It is because of the overwhelming feedback that alpha and beta testers provide. When you read the Linux kernel, you will find many parts are poorly structured, poorly written and poorly documented. However, people dared to test it and report their problems; Linus and friends respected the error reports and went ahead to fix them. That is why it works so well.

In addition, psychology sometimes causes weird effects. If a user discovers a bug in his system, reports the bug and sees it fixed eventually, that user is happy because he was treated with respect. Most likely, he is even happier than he would be in the bug-free case.

We not only need to bring the CREDITS file into an accurate state, but we also need to acknowledge the thousands who have contributed to Linux by using it and sharing their discoveries—good or bad—with others.

Peter H. Salus reports the UNIX philosophy in A Quarter Century of UNIX as:

  • Write programs to work together.

  • Write programs that handle text streams, because that is a universal interface.

I'd like to close by adding another entry, suggested by UNIX and dominant in Linux:

  • Write programs you enjoy.

Postscript

I just received a note from the person who sparked the original survey. He reports:

I took my “hand-me-down” Linux box, an unimpressive 75MHz Pentium with 64MB RAM and a tiny 600MB HD to work. My boss was amazed that office applications such as StarOffice were available and was quite impressed when I read a Word document with StarOffice and then converted it to HTML. Samba was another revelation. Overall performance impressed him. In a few crude tests, it outperformed a “commercial” system running with 128MB RAM, dual 200MHz processors and all ultra-fast/ultra-wide SCSI drives.

After a couple of callers indicated an interest in UNIX versions, we checked the price of current systems. My boss decided Linux was indeed priced right, and asked me to start on a port.

It looks like we've won one more away from the dark side.

Randolph Bentson 's first UNIX experience was booting a BSD VAX system on July 3, 1981—the whole town had a celebration the next day. Dr. Bentson started contributing to the Linux kernel in May 1994, and his book Inside Linux: A Look at Operating System Development describes how many modern operating system features have evolved and become essential parts of Linux. He can be reached at bentson@grieg.seaslug.org.