Fixing problems of centralization with more centralized systems only makes the problem worse.
The Net as we know it today first became visible to me in March 1994, when I was among several hundred other tech types gathered at Esther Dyson's PC Forum conference in Arizona. On stage was John Gage (en.wikipedia.org/wiki/John_Gage) of Sun Microsystems, projecting a Mosaic Web browser (en.wikipedia.org/wiki/Mosaic_(web_browser)) from a flaky Macintosh Duo (en.wikipedia.org/wiki/PowerBook_Duo), identical to the one on my lap. His access was to Sun over dial-up.
Everybody in the audience knew about the Net, and some of us had been on it one way or another, but few of us had seen it in the fullness John demonstrated there. (At that date, there were a sum total of just three Internet Service Providers.) James Fallows (www.theatlantic.com/james-fallows) was in the crowd, and he described it this way (listserv.aera.net/scripts/wa.exe?A2=ind9406&L=aera-f&D=0&P=351) for The Atlantic:
In the past year millions of people have heard about the Internet, but few people outside academia or the computer industry have had a clear idea of what it is or how it works. The Internet is, in effect, a way of combining computers all over the world into one big computer, which you seemingly control from your desk. When connected to the Internet, you can boldly prowl through computers in Singapore, Buenos Aires, and Seattle as if their contents resided on your own machine.
In the most riveting presentation of the conference, John Gage, of Sun Microsystems, demonstrated the World Wide Web, the gee-whizziest portion of the Internet, in which electronic files contain not only text but also graphics and sound and video clips. Using Mosaic, a free piece of “navigator” software that made moving around the Web possible, Gage clicked on icons on his screen exactly as if he were choosing programs or directories on his own hard disk. He quickly connected to a Norwegian computer center that had been collecting results during the Winter Olympics in Lillehammer and checked out a score, duplicating what Internet users had done by the millions every day during the games, when CBS-TV was notoriously late and America-centric in reporting results.
Note the terms here. John used Mosaic to “control”, “boldly prowl” and “navigate” his way around the Web, which was the “gee-whizziest portion” of the Net.
That portion has since become conflated with the whole thing. Today we use browsers to do far more than navigate the Web. Protocols that once required separate apps—file transfer, e-mail, instant messaging—are now handled by browsers as well. We now also can use browsers to watch television, listen to radio and read publications. It's hard to name anything a computer can do that isn't also doable (and done) in a browser. Serving up most of those capabilities are utility Web services, provided by Amazon, Apple, Dropbox, Evernote, Google, Yahoo and many more, each with their own clouds. The growth of the Web, atop the Net, also has provided a conceptual bridge from computers to smartphones and tablets. Today nearly every mobile app would be useless without a back-end cloud.
While relying on the Web and its clouds has increased the range of things we can do on the Net, our freedom to act independently has declined. The browser that started out as a car on the “information superhighway” has become a shopping cart that gets re-skinned with every commercial site it visits, carrying away tracking beacons that report our activities back to centralized servers over which we have little if any control. The wizards among us might be adept at maintaining some degree of liberty from surveillance, but most muggles are either clueless about the risks or make do with advertising and tracking blockers. This is less easy in the mobile world, where apps are more rented than owned, and most are maintained by vendor-side services.
Thus, we've traded our freedom for the conveniences of centralization. The cure for that is decentralization: making the Net personal, like it promised to be in the first place—and still is, deep down.
It should help to remember that the Web is polycentric while the Net is decentralized. By polycentric, I mean server-based: every server is a center. So, even though Tim Berners-Lee wanted the Web to be what he called “a distributed hypertext system” for “universal linked information” (www.w3.org/History/1989/proposal.html), what he designed was servers “generating a hypertext representation”, as shown in Figure 1.
Figure 1. Servers Generating a Hypertext Representation
Today this looks like your e-mail on a Google server—or your photos on Instagram or your tweets on Twitter. There's nothing wrong with any of those, just something missing: your independence and autonomy.
Meanwhile, the Net beneath the Web remains decentralized: a World of Ends (worldofends.com) in which every end is a functional distance of zero from every other end. “The end-to-end principle is the core architectural guideline of the Internet” says RFC 3724. Thus, even though the Internet is a “collection of networks”, what collects them are the transcendent purposes of the Net's ends, which consist of you, me, Google and every other node.
If you want to grok the problems of centralization fully, and their threat to personal freedom, to innovation and to much else, watch, listen to or read Eben Moglen's lectures titled “Snowden and the Future” (snowdenandthefuture.info), given in November and December 2013 at Columbia University, where Eben has been teaching law for 26 years. The lectures are biblical in tone and carry great moral weight. For us in the Linux community, they are now in the canon.
What Eben calls for is not merely to suffer the problems of centralization, but to solve them. This requires separating the Net and the Web. For me, it helps to think of the Net as the ground we walk and drive on, and the Web as clouds in the sky, as I've illustrated with the photo in Figure 2.
Figure 2. It helps to think of the Net as the ground we walk and drive on, and the Web as clouds in the sky.
There are many possibilities for decentralized solutions on the Net's ground, and I hope readers will remind us of some. Meanwhile, I'll volunteer a pair I've been watching lately. One is TeleHash, and the other is XDI.
TeleHash (telehash.org) is the brainchild of Jeremie Miller, father of Jabber and the XMPP protocol for instant messaging. Its slogan is “JSON + UDP + DHT = Freedom”, and it is described as “a new wire protocol enabling applications to connect privately in a real-time and fully distributed manner, freeing them from relying on centralized data centers”. The rest of the index page says:
What
It works by sending and receiving small encrypted bits of JSON (with optional binary payloads) via UDP using an efficient routing system based on Kademlia (en.wikipedia.org/wiki/Kademlia), a proven and popular Distributed Hash Table.
Demo
It's very much in the R&D stages yet, but check out hash-im (https://github.com/quartzjer/hash-im) for a simple demo.
Status
The current spec (https://github.com/telehash/telehash.org/blob/master/protocol.md) is implemented in a few languages (any help here would be great!), and prototype apps are being created to test it. Questions can be directed at Twitter (https://twitter.com/jeremie), or to Jeremie Miller directly.
XDI (xdi.org) is a mostly-baked standard. Its purpose is “to define a generalized, extensible service for sharing, linking, and synchronizing data over digital networks using structured data formats (such as JSON and XML) and XRIs (Extensible Resource Identifiers), a URI-compatible abstract identifier scheme defined by the OASIS XRI Technical Committee” (https://www.oasis-open.org/committees/tc_home.php?wg_abbrev=xdi). Wikipedia (at the moment) says (en.wikipedia.org/wiki/XDI):
The main features of XDI are: the ability to link and nest RDF graphs to provide context; full addressability of all nodes in the graph at any level of context; representation of XDI operations as graph statements so authorization can be built into the graph (a feature called XDI link contracts); standard serialization formats including JSON and XML; and a simple ontology language for defining shared semantics using XDI dictionary services.
XDI graphs can be serialized in a number of formats, including XML and JSON. Since XDI documents are already fully structured, XML adds very little value, so JSON is the preferred serialization format. The XDI protocol can be bound to multiple transport protocols. The XDI TC is defining bindings to HTTP and HTTPS, however it is also exploring bindings to XMPP and potentially directly to TCP/IP.
XDI provides a standardized portable authorization format called XDI link contracts (en.wikipedia.org/wiki/Link_contract). Link contracts are themselves XDI documents (which may be contained in other XDI documents) that enable control over the authority, security, privacy, and rights of shared data to be expressed in a standard machine-readable format and understood by any XDI endpoint.
This approach to a globally distributed data sharing network models the real-world mechanism of social contracts (en.wikipedia.org/wiki/Social_contract), and legal contracts that bind civilized people and organizations in the real world today. Thus, XDI can be a key enabler of the Social Web (en.wikipedia.org/wiki/Social_Web). It has also been cited as a mechanism to support a new legal concept, Virtual Rights (www.virtualrights.org), which are based on a new legal entity, the “virtual identity”, and a new fundamental right: “to have or not to have a virtual identity”.
It's early for both of these. But I know in both cases the mentality of the developers is on the ground of the Net and not lost in the clouds of the Web. We'll need a lot more of that before we all get our freedom back.