Larry Wall, the Guru of Perl

I “talked” to Larry Wall, the creator of the Perl scripting language, by e-mail on March 1. Larry proved to be quite voluble, and I think you'll find this interview fun as well as informative. I certainly did.

Marjorie: Back in the beginning, what inspired you to write Perl?

Larry: That depends on what you mean by “beginning”. Like Moses said: “In the beginning, God created the heavens and the earth.” I'm not being entirely facetious about that. Whichever way you care (or don't care) to interpret scripture, I think the universe is a pretty hefty inspiration for anyone who aspires to be a creator. I've certainly tried to put a universe of ideas into Perl, with some amount of success.

In terms of biographical beginnings, my father was a pastor, as were both my grandfathers, and many of my ancestors before that. My wife likes to say that preachers are bred for intelligence (though I suppose she might be saying that just to flatter me). Be that as it may, I did receive a fairly decent set of brain construction genes. Beyond that, I also received a rich heritage of ideas and skills, some of which found their way into Perl culture. For instance, the notion that you can change the world. The idea that other people are important. The love of communication and an understanding of rhetoric, not to mention linguistics. The appreciation of the importance of text. The desire to relate everything to everything else. The passion to build up rather than tear down. And, of course, the dead certainty that true wealth is measured not by what you accumulate, but by what you pass on to others.

The beginnings of Perl were directly inspired by running into a problem I couldn't solve with the tools I had. Or rather, that I couldn't easily solve. As the Apostle Paul so succinctly put it, “All things are possible, but not all things are expedient.” I could have solved my problem with awk and shell eventually, but I possess a fortuitous surplus of the three chief virtues of a programmer: Laziness, Impatience and Hubris. I was too lazy to do it in awk because it would have been hard to get awk to jump through the hoops I was wanting it to jump through. I was too impatient to wait for awk to finish because it was so slow. And finally, I had the hubris to think I could do better.

Of course, actually writing something like Perl takes a great deal of hard work, patience and even humility. Had I just been doing it for myself, I probably wouldn't have made the effort. However, I was aware from the beginning that other people were going to be using Perl, so I've always integrated the “laziness curve” over the whole community, not just over myself. I was being vicariously lazy. So here we are talking about vicars again.

Marjorie: Well, that certainly answered the question fully. I must admit I didn't expect you to go back as far as the beginning of the Universe. :-) How'd you come up with that name?

Larry: I wanted a short name with positive connotations. (I would never name a language “Scheme” or “Python”, for instance.) I actually looked at every three- and four-letter word in the dictionary and rejected them all. I briefly toyed with the idea of naming it after my wife, Gloria, but that promised to be confusing on the domestic front. Eventually I came up with the name “pearl”, with the gloss Practical Extraction and Report Language. The “a” was still in the name when I made that one up. But I heard rumors of some obscure graphics language named “pearl”, so I shortened it to “perl”. (The “a” had already disappeared by the time I gave Perl its alternate gloss, Pathologically Eclectic Rubbish Lister.)

Another interesting tidbit is that the name “perl” wasn't capitalized at first. UNIX was still very much a lower-case-only OS at the time. In fact, I think you could call it an anti-upper-case OS. It's a bit like the folks who start posting on the Net and affect not to capitalize anything. Eventually, most of them come back to the point where they realize occasional capitalization is useful for efficient communication. In Perl's case, we realized about the time of Perl 4 that it was useful to distinguish between “perl” the program and “Perl” the language. If you find a first edition of the Camel Book, you'll see that the title was Programming perl, with a small “p”. Nowadays, the title is Programming Perl.

Marjorie: Okay, is Perl perfect now or do you continue to do further development?

Larry: Hmm, the two are not mutually exclusive. Look at Linux. :-)

Actually, Perl was never designed to be perfect. It was designed to evolve, to become more adaptive, as they say. There is no such thing as a perfect organism, biologically speaking. About the most you can say is an organism is more or less suited for the environment in which it finds itself. In fact, biologists are just now realizing that any organism which seems to be “perfect” for one environment is likely to be in danger of extinction as soon as the environment changes. Over-specialization is only as good as your ecological niche. We're not just talking about dinosaurs here, but also snail darters and cheetahs and a bazillion beetles in Brazil—not to mention Visual Basic.

We've already seen the deaths of many over-specialized organisms in computing: Lisp machines, Ada chips and many so-called fourth generation languages. Any program ever written in assembly language for an obsolete architecture is now obsolete. Likewise, any program that ties its fortunes to a single operating system is likely to go down with the ship. I don't know how many more torpedoes Windows can take before it sinks, but if and when it does, a whole batch of specialized programs are going down with it. Obviously, for reasons relating to the open source movement, Linux doesn't have this particular problem.

Anyway, back to Perl. Right from the start, Perl was designed for change. This involves certain tradeoffs, some of which appear to be suboptimal to people who don't think the way I think. For instance, I wanted to be able to add new keywords to Perl without breaking old programs, so I put them into a separate namespace from variable names. This meant either variable names or keywords had to be marked somehow as special. I chose to mark variables, since it also made it easy to interpolate variables into strings, and since there's a history of marking variables in computer languages such as BASIC. Note this was actually non-adaptive in certain environments, namely in the minds of certain purists who think the added punctuation makes Perl ugly, and too much like BASIC. Well, maybe it does. So what? That was a conscious tradeoff so that Perl would be more useful in the long run. In that respect, Perl is less adaptive in the specific ecological niche comprising the minds of computer scientists, but more adaptive in the world as a whole. I've never regretted that particular tradeoff.

Of course, once you get past first impressions, there are many things in Perl that computer scientists do like, such as lexically scoped variables and closures. So by and large, those computer scientists who can hold their nose long enough to get the cheese into their mouths find the taste bearable.

More importantly, Perl 5 introduced an extension system that, much like Linux's module system, allows continued development of the language without actually changing the core language. That is, you can pull in a Perl module that warps the language to your purposes in a controlled fashion. If a module becomes popular enough, we can consider making it part of the core of Perl—maybe.

That's not to say we never change the core anymore. We recently added support for multi-threading and for Unicode. Interestingly, even when we do make changes to the core these days, we make it look as though the programmer is pulling in an extension module. Essentially, if you use a fancy new core feature that warps the semantics in some way, you have to declare it. This is how we maintain almost complete compatibility with older Perl scripts. Most Perl 1 scripts still run unchanged under Perl 5. As a side benefit, feature declarations are right up front where the dependency is visible at compile time, so we rarely die in the middle of execution for lack of a feature. Compare this with shell programming, where you don't even know whether all the programs you're intending to invoke actually exist until you try to run them, and then, kablooey!

Marjorie: What are your future development plans for Perl?

Larry: If I could predict that, I'd be a smarter person than I am. I'm just smart enough to know I'm no smarter than that, which is why I designed Perl to evolve in the first place.

That being said, I can tell you some of the characteristics I look for in a project.

First, if it has anything to do with text processing, Perl is a natural. Perl has never stopped being a text-processing language, though it long ago escaped the straitjacket of being just a text processing language. That's one reason Perl was a natural for CGI programming, because Perl excels at ripping text apart and putting it back together.

Second, I look for projects that involve gluing things together. We don't use glue on Legos—we glue together things that weren't designed to go together. As a glue language, Perl has thin characteristics so that it can flow into tiny gaps, and thick characteristics so that it can fill in larger voids. Perl is always at home in the interstices. The typical CGI script or mod_perl servlet glues a database together with the Web. When that particular interstice disappears, there will be other interstices.

Third, I look for projects that franchise the disenfranchised. We joke about sending our leftovers to the starving people in Africa, but there are, in fact, billions of potential programmers outside of America who can't afford to lay out hundreds of bucks for an operating system or an application. China recently put in a single order for 200,000 Internet books from a publisher I know (and work for). That's just the beginning. This is why I hacked Unicode support into Perl last year. Of course, text processing has something to do with Unicode too.

Having said all that, it almost doesn't matter what I look for in my next development project, because I don't do most of the Perl development these days. The Perl community outweighs me by many orders of magnitude, and they're really the ones who are making Perl the be-all and end-all of scripting languages. I just sit on the sidelines and cheer occasionally. I'm cheering now. Rah, rah, rah! :-)

Marjorie: In what way is Perl better than other scripting languages such as Python and Eiffel?

Larry: Perl is unique, not just among scripting languages, but among computer languages in general. It's the only computer language consciously and explicitly designed to be postmodern. All other computer languages are still stuck in the modern era to some degree. Now, as it happens, I don't normally use the term “postmodern” to describe Perl, because most people don't really understand postmodernism, even as they embrace it. But the fact is that American culture has become thoroughly postmodern, not just in music and literature, but also in fashion, architecture and in overall multicultural awareness.

Modernism was based on a kind of arrogance, a set of monocultural blinders that elevated originality above all else, and led designers to believe that if they thought of something cool, it must be considered universally cool. That is, if something's worth doing, it's worth driving into the ground to the exclusion of all other approaches. Look at the use of parentheses in Lisp or the use of white space as syntax in Python. Or the mandatory use of objects in many languages, including Java. All of these are ways of taking freedom away from the end user “for their own good”. They're just versions of Orwell's Newspeak, in which it's impossible to think bad thoughts. We escaped from the fashion police in the 1970s, but many programmers are still slaves of the cyber police.

In contrast, postmodernism allows for cultural and personal context in the interpretation of any work of art. How you dress is your business. It's the origin of the Perl slogan: “There's More Than One Way To Do It!” The reason Perl gives you more than one way to do anything is this: I truly believe computer programmers want to be creative, and they may have many different reasons for wanting to write code a particular way. What you choose to optimize for is your concern, not mine. I just supply the paint—you paint the picture.

Marjorie: Who is using Perl and how are they using it?

Larry: A couple of years ago, I ran into someone at a trade show who was representing the NSA (National Security Agency). He mentioned to someone else in passing that he'd written a filter program in Perl, so without telling him who I was, I asked him if I could tell people that the NSA uses Perl. His response was, “Doesn't everyone?” So now I don't tell people the NSA uses Perl. I merely tell people the NSA thinks everyone uses Perl. They should know, after all.

As an interesting side note, it turned out this fellow was the very administrator who shut down the NSA project Perl was (indirectly) written to support. He was vaguely amused when I pointed out Perl might well be the most enduring legacy of the project.

As to what everyone uses Perl for, it's really all over the map. I was astounded several years ago to be told how heavily Perl is used on Wall Street. “A Perl book on every other desk” is how I heard it. But it makes sense when you realize that market analysts need to revise their models continually, and they need to scan news services for information that might be related to their positions in the market. Rapid prototyping and text processing are what they need.

Many people associate Perl with CGI scripts, though of course most of the heavy lifting is done with mod_perl servlets under Apache. Perl is used just as much on the client side in the robots and spiders that navigate the Web and build much of the linkage implicit in various on-line databases. And that's not all. If you've ever been spammed (and who hasn't?), your e-mail address was almost certainly gleaned from the Net using a Perl script. The spam itself was likely sent via a Perl script. One could say that Perl is the language of choice for Net abuse. And one could almost be proud of it.

That's only scratching the surface of what Perl is used for. Without getting Mr. Gallup or the U.S. Census Bureau involved, the best way to figure out what Perl is used for is to look at the 800 or so reusable extension modules in the Comprehensive Perl Archive Network (the CPAN, for short). If you glance through those modules, you'll get the impression that Perl has interfaces to almost everything in the world. With a little thought, you may figure out the reason Perl has interfaces to everything is not so much so Perl itself can talk to everything, but so Perl can get everything in the world talking to everything else in the world. The combinatorics are staggering. The very first issue of The Perl Journal (not to be confused with Linux Journal) contained an article entitled “How Perl Saved the Human Genome Project”. It explains how all the different genome sequencing laboratories used different databases with different formats, and how Perl was used to massage the data into a cohesive whole.

Marjorie: We received a product announcement for PerlDirect from ActiveState Tool Corporation that says:

Are you affiliated with this company? I think it's interesting they are offering to let a subscriber have direct input to open-source development for $12,000 a year. Does this make sense?

Larry: Sounds like a pretty ordinary support contract to me. I don't think even Richard Stallman would disagree with the notion that support is a valid way to make money off free software.

I'm not directly affiliated with ActiveState, but I've worked with them, and I think the problems they've solved far outweigh any problems they've created. You've got to understand their market has always been the Windows space, where you're actually doing people a favor by charging them money for things, because that's the only way to keep from confusing them. Linux users are smarter than this, of course, but some Linux users aren't quite smart enough to realize Windows is a different culture, and Perl, being a postmodern language that is sensitive to context, will look different in a different culture.

Marjorie: Oops, didn't mean to sound as if I thought they weren't on the up-and-up—just curious if you knew them. What are your views of the Open Source movement? Do you think it will become a true phenomenon, or is it just a passing fancy?

Larry: I must have a conjunctive rather than a disjunctive brain, because I think both of those notions are true. And I also think they're both false. :-)

How can we claim open source is becoming a true phenomenon when it has already been a true phenomenon for a couple of decades now? We're merely pointing out to everyone a practice that has a proven track record of producing excellent code. On the other hand, we're certainly trying to make it a truer phenomenon, in the sense that we hope more people will feel it's a valid development model for many kinds of software that were formerly developed under a closed model.

And, of course, it's a passing fancy—just as we've had other passing fancies for free-form syntax, structured programming, and more recently, object orientation.

What you have to understand is that, from the viewpoint of the passing fancy, people represent a kind of fork in the road. It's like separating the sheep from the goats in the book of Revelation. Some of these fancies pass on the one side and go into oblivion, while others pass on the other side and go into common practice, usually after a period of excessive enthusiasm. Free-form syntax, structured programming and object orientation are all good (in moderation). But note that all of these passing fancies had a history of being useful before they became popular. The passing fancies that go into oblivion are the ones rooted not in history, but rather in someone's wishful thinking (usually someone from marketing). By this criterion, open source will probably go into common practice because it's already in common practice.

The way I see it, the open source movement is just another manifestation of the growing postmodernism of our culture. By contrast, the notion of trade secrecy is just a rehash of the modernistic idea of originality at any cost. We've had a lot of lip service given to code reuse over the years, but it really only works with open source. A postmodern computer programmer truly believes in reusing good code whether it's original or not. It's not a point of pride anymore. A good postmodern is supposed to plagiarize the things he or she thinks are cool.

Marjorie: If everything becomes open source, how will programmers make a living?

Larry: Contrary to many open source advocates, I don't see everything becoming open source. What I do see is a growing recognition that anything resembling large-scale infrastructure ought to be open source, much as the United States has recognized that interstate highways should not be toll roads. On the other hand, we don't expect city parking lots to be free except in certain enlightened municipalities. So I'd expect to see Windows become open source before Word does.

That being said, there are many ways to make a living off open source, just as there are many ways to make a living off open science. But here's precisely where I think the open source movement has some growing to do. Open science basically started out as a hobby of the rich, but it didn't truly blossom into the form we recognize today until its patronage was taken over by educational institutions. This has not quite happened yet with open source. Or rather, it started to happen, but then many educational institutions got caught up in the drive for the almighty dollar. I wish more places would follow the example of UC Berkeley.

Marjorie: On that note, how do you make a living?

Larry: To start off, I worked programming and sys admin jobs like anyone else and did my free software on the side. Later, I wrote a book and started collecting royalties. The book became a best-seller, so it made my publisher, O'Reilly & Associates, even more money than it made me. Of course, they have to pay more people with that money, so it evens out.

Anyway, three years ago it occurred to Tim O'Reilly and me that anything good for Perl was also good for O'Reilly & Associates, so now they pay me to do whatever I like, as long as it helps Perl. It's been a good symbiosis.

Marjorie: Any interesting projects you'd like to tell us about?

Larry: I'm supposed to be working on the third edition of the Camel Book, so I don't officially have any other interesting projects at the moment. Of course, I have been playing around with that Palm Pilot Tim O'Reilly gave me for Christmas, but I won't tell if you won't.

Marjorie: Agreed. Give us some personal info—where you went to school, interests, etc.

Larry: I spent the first half my childhood in south Los Angeles about two miles from where the Watts riots broke out, and the second half of my childhood in Bremerton, Washington, where no riots broke out, but I did graduate from high school. For the third half of my childhood, I went to Seattle Pacific University, where I started off majoring in chemistry and music, later switched to premed, and eventually (after taking several years off to work in the SPU computer center) ended up majoring in Natural and Artificial Languages (a self-designed major). After that, my wife and I attended grad school in linguistics at Berkeley and UCLA. At the time, we were actually planning to be missionaries (more specifically, Bible translators), but we had to drop that idea for health reasons. Funny thing is, now the missionaries probably get more good out of Perl than they'd have gotten out of me as a missionary. Go figure.

As for my interests, that's hard, because I tend to be interested in anything that's interesting. Which comes out to pretty much anything except opera and soap opera—space opera's okay, though.

Marjorie: What do you do for fun?

Larry: Read and listen to my wife read to me (especially space opera). Discuss anything and everything with anyone and everyone in my family. Work NY Times crossword puzzles. Play mah jong. Practice aikido. Watch anime. (Maybe Japanese soap opera is okay.) Play with my fish. Rescue my fish from the equipment I bought to keep them alive.

Marjorie: Looks like you stay busy and have fun—a good combination. What do you eat for breakfast?

Larry: I eat all kinds of things for breakfast—but then, I generally eat breakfast at lunchtime.

Marjorie: Seems as good a time as any. Thanks for giving us so much of your time. It's been interesting.