We've built and re-built centralized top-down systems for the duration. Time for something that's not.
One of the great works of geekdom is Neal Stephenson's In the Beginning Was the Command Line, an essay-length book that came out in 1999 (www.cryptonomicon.com/beginning.html). As with Linux, the code was open. Still is. Here's one copy of the book's full text: cristal.inria.fr/%7Eweis/info/commandline.html. Though many of Neal's references (for example, the Be operating system) are forgotten or stale, his case for Linux (and its UNIX relatives) is as fresh and right as ever. Here is the gist of it:
The file systems of Unix machines all have the same general structure. On your flimsy operating systems, you can create directories (folders) and give them names like Frodo or My Stuff and put them pretty much anywhere you like. But under Unix the highest level—the root—of the filesystem is always designated with the single character “/” and it always contains the same set of top-level directories:
/usr
/etc
/var
/bin
/proc
/boot
/home
/root
/sbin
/dev
/lib
/tmp
and each of these directories typically has its own distinct structure of subdirectories. Note the obsessive use of abbreviations and avoidance of capital letters; this is a system invented by people to whom repetitive stress disorder is what black lung is to miners. Long names get worn down to three-letter nubbins, like stones smoothed by a river.
This is not the place to try to explain why each of the above directories exists, and what is contained in it. At first it all seems obscure; worse, it seems deliberately obscure. When I started using Linux I was accustomed to being able to create directories wherever I wanted and to give them whatever names struck my fancy. Under Unix you are free to do that, of course (you are free to do anything), but as you gain experience with the system you come to understand that the directories listed above were created for the best of reasons and that your life will be much easier if you follow along (within /home, by the way, you have pretty much unlimited freedom).
After this kind of thing has happened several hundred or thousand times, the hacker understands why Unix is the way it is, and agrees that it wouldn't be the same any other way. It is this sort of acculturation that gives Unix hackers their confidence in the system, and the attitude of calm, unshakable, annoying superiority captured in the Dilbert cartoon. Windows 95 and MacOS are products, contrived by engineers in the service of specific companies. Unix, by contrast, is not so much a product as it is a painstakingly compiled oral history of the hacker subculture. It is our Gilgamesh epic.
What made old epics like Gilgamesh so powerful and so long-lived was that they were living bodies of narrative that many people knew by heart, and told over and over again—making their own personal embellishments whenever it struck their fancy. The bad embellishments were shouted down, the good ones picked up by others, polished, improved, and, over time, incorporated into the story. Likewise, Unix is known, loved, and understood by so many hackers that it can be re-created from scratch whenever someone needs it. This is very difficult to understand for people who are accustomed to thinking of OSes as things that absolutely have to be bought.
When Tim Berners-Lee invented the World Wide Web, he did it on a NeXT machine, which also had UNIX ancestors. But never mind that. What matters is that Tim arranged the web's directory on the UNIX model: files were nested in directories divided hierarchically by a series of slashes: http://example.com/this/that/etc. The same hierarchy is also implicit in client-server, which should have been called server-client, to reflect the arrangement of status: server over client.
Between the directory path and client-server, what we've got is a mental model of computing and networking that anchors all of computing's civilization.
But it also gives us problems when it comes to conceiving and designing distributed approaches, such as peer-to-peer. It gives us hierarchy after hierarchy, and hierarchies within hierarchies, rather than the heterarchy that Paul Baran imagined the future internet to embody when he drew his diagram in 1964 (Figure 1).
Figure 1. Centralized, Decentralized and Distributed Networks
While we're at it, let's also revisit “End-To-End Arguments in System Design” (www.ece.ucdavis.edu/%7Echuah/classes/eec273/eec273-w12/refs/SRC84-e2e.pdf) by David P. Reed, Jerome H. Saltzer and David D. Clark: a design guide (web.mit.edu/Saltzer/www/publications/endtoend/ANe2ecomment.html) that helped manifest and rationalize the internet protocol (https://en.wikipedia.org/wiki/Internet_Protocol), which in an ideal world would give us a fully distributed network (C, in Figure 1). Alas, the current internet's implementation is closer to decentralized (B, in Figure 1). As distributed networks go, it's good, but not good enough, because it still subordinates client nodes to server ones, so servers get scale, while clients get little more scale than the servers (and the operators of sphincters in the network itself) allow. We also have a networked world where governments can favor or ban traffic they don't like. Even society itself has to some degree been re-organized and re-centralized by giant private “social networks”, such as Facebook and LinkedIn.
Back in April 2016, I suggested that our next fight is for freedom from exactly the kind of centralized systems I just described (www.linuxjournal.com/content/whats-our-next-fight). Fighting for freedom would also get us closer to each of these ideals:
General-purpose computing and networking.
Decentralization and distributed everything.
Privacy.
The true Internet of Things.
Now I suggest that we also need to free ourselves from the very mental models that we used to build giant centralized traps from which we need to escape.
There are positive signs. The blockchain, for all its faults, is distributed by design. To come up with blockchain and Bitcoin (which uses a blockchain), Satoshi Nakamoto (or whoever that really is) had to think outside of fiat currency, banks, centralized trust systems and the other familiar boxes that control transactions in the world's economies, nearly all of which are centralized by design. He had to think of ways that a fully distributed peer-to-peer approach to all those things would open possibilities and outperform currency, payments and record-keeping done the old ways. One can criticize Bitcoin and blockchain on many grounds, but what matters is that a distributed system got imagined and implemented causing many other new re-thinks and re-builds in the world.
The problem with mental models is that they work—also that we can't escape the deepest ones that are anchored in our own experience as physical bodies, operating in the physical world.
Ever wonder why good is up (or high) and bad is down (or low)? Why it is easier to conceive of heaven in the sky and hell below the ground, than vice versa? Or why light is good and dark is bad? Or why people say “enlightened” rather than “endarkened?” Or why we “catch” or “grasp” ideas? The answer is, because we are upright-walking diurnal animals with hands and opposable thumbs. If owls or moles were equipped by nature with the means to have moral systems and speak about them, their metaphors would be radically different. Dark might be good, and light might be bad.
We are embodied animals, and we can't get away from that fact. But we are also inherently distributed, and different. At a base level, we are heterozygous. No two of us are the same, unless we are identical twins; and even then we are separate and distinct individuals. (An interesting fact: so are apples. Writes Michael Pollan in The Botany of Desire, every seed in every apple “contains the genetic instructions for a completely new and different apple tree, one that, if planted, will bear only the most glancing resemblance to its parents”. All the varieties of Apple we know—Granny Smith, Delicious, Macintosh—grow on trees that start as grafts off a single ancestral plant.)
The designs we need are ones that appreciate our heterozygous inheritances, and the fact that we are designed to learn throughout our healthy lives. “Encompass worlds, but never try to encompass me”, Walt Whitman advises. He adds:
Urge and urge and urge, Always the procreant urge of the world. Out of the dimness opposite equals advance. Always substance and increase, always sex, Always a knit of identity, always distinction, Always a breed of life.
I love how Whitman puts those together, because none of them fits in a system, other than one even he fails to comprehend, even as he embraces its mystery. I also love “knit of identity”, because each life is a thread distinct in its substance and capacity for increase, yet part of a whole that changes as well. Every self, like every species, is a breed of life.
It is hard for computing to comprehend this, but not for the minds of people programming and using computers.
Computing and programming require that we think of both in explicit ways, and in explicit terms. Yet our knowledge of the world is mostly tacit. “We know more than we can tell”, Michael Polanyi says, and that's a near absolute understatement. It applies to everything we think and say. For example: even if I've made full sense to you in this column so far, you probably won't be able to repeat it back to me verbatim. And if you could, it would owe more to memorization than comprehension. Short-term memory is an amazing grace of human nature. It forces us to communicate meaning more than words. Consider how often, in the midst of explaining something, we don't remember exactly how we started the sentences we are now speaking, or exactly how we will finish them, yet somehow we'll say what we mean, and others will understand it, even though they can't repeat exactly what we said.
That's because, when we communicate with each other, we don't deliver an explicit cache of words. Instead we cause meaning to form in the mind of another person. Meaning is most of what we take away from any conversation. The same goes for any course in school, any book or any experience. The meaning we take is mostly tacit. It is also mostly unquestioned, once we make it our own.
Here's how I put it many years ago in a chapter of Open Sources 2.0 (programmer.97things.oreilly.com/wiki/index.php/Open_Sources_2.0/Beyond_Open_Source:_Collaboration_and_Community/Making_a_New_World):
Several years ago I was talking with Tim O'Reilly about the discomfort we both felt about treating information as a commodity. It seemed to us that information was something more than, and quite different from, the communicable form of knowledge. It was not a commodity, exactly, and was insulted by the generality we call “content”.
Information, we observed, is derived from the verb inform, which is related to the verb form. To inform is not to “deliver information”, but rather, to form the other party. If you tell me something I didn't know before, I am changed by that. If I believe you and value what you say, I have granted you authority, meaning I have given you the right to author what I know.
Therefore, we are all authors of each other. This is a profoundly human condition in any case, but it is an especially important aspect of the open-source value system. By forming each other, as we also form useful software, we are making the world, not merely changing it.
So now look at authoring as something all of us do—or should be able to do—all the time, in many more ways and contexts than our pyramidal centralized systems would allow.
Consider the possible purposes of both our heterogeneousness and our enormous capacity to communicate and learn, throughout our lives. Why are we that way? Are those very human natures not insulted by systems built to subordinate individuality to categories in databases? Is the full promise of heterarchy (www.linuxjournal.com/content/opening-minds-spheres-among-us) not a price we pay for making nothing but hierarchies, over and over again, because that's what our tools and mental models are biased to do?
That we come in many colors, sizes and body shapes—all with different faces that also change as we grow and age—is a grace meant to help us recognize every person as distinctive and separate. Not just so we can typify each other by any one of those characteristics. None of us is just black or white, male or female, tall or short. We are sovereign selves with complete souls that cannot be reduced to any one characteristic, no matter how easy it is to do that, especially with research and computers.
I bring this up because I believe it is also worth considering that the best case for distributed systems and networks is that they take advantage of the countless differences and originalities among us. Distributed systems, more than any other kind we can name—make possible recognizing that our greatest resources are each other—and ourselves.