EOF

Searching has become shopping, whether we like it or not. That's the assumption behind the results, and behind recent changes, at least to Google's search features and algorithms. I'm sure this isn't what Google thinks, even though it is a commercial enterprise and makes most of its money from advertising—especially on searches. Credit where due: from the moment it started adding advertising to search results, Google has been careful to distinguish between paid results and what have come to be called “organic”.

But alas, the Web is now a vast urban scape of storefronts and pitch-mills. The ratio of commercial to noncommercial sites has become so lopsided that Tim Berners-Lee's library-like World Wide Web of linked documents has been reduced to a collection of city parks, the largest of which is Wikipedia (which, like Google, runs on Linux, according to Netcraft). Without Wikipedia, organic results for many subjects would be pushed off the front page entirely.

Let's say you want to know more about the common geometric figure called a square. (Or that you'd like to explore one of the other organic meanings of square, such as correct, level, honest or retro.) You can find your answers using Google or Bing, but none will be the top “I'm feeling lucky” result if you just search for “square” alone. Instead, you'll get the name of a company.

For example, when I look up square on my Chrome browser while also logged in to my Google account, the engine produces a crufty URL more than 200 characters long. The top result for this search is the company Square.com. With that result, Google also tells me Robert Scoble gives the company a “+1”, and that I have “110 personal results and 296,000,000 other results”. Google's engine is now tweaked to think I might want those results because I've shared a bunch of personal data in the course of whatever I do in Google's presence. But, while I think Square.com is a fine and interesting company, that's not the result I'm looking for.

When I carve the cruft out of that URL (so it just says https://www.google.com/search?en&q=square), the result is the same, without the bonus item from Scoble or the other personalized “social” stuff. When I search on Bing (with which I have no logged-in relationship), I also get long a crufty URL, again with Square.com as the top result. Likewise, if I reduce Bing's URL to just http://www.bing.com/search?q=square, I'm still at square zero (pun intended).

In both search engines, the #2 or #3 result is Wikipedia's one for the geometric square. So, you might say, the systems work in any case: they just happen to make Square.com the top result, because page ranking is based mostly on inbound links, and Square.com gets more of those than any page that describes the geometric shape. And hey, if you're looking for “taxi” on the streets of New York, you're probably looking for a business with four wheels, not for a definition of one.

Search engines operate today in the Web's urban environment, which is at least as commercial as any brick-and-mortar one. Search engines are also not libraries, even if libraries share Google's mission of “organizing the world's information and making it accessible and useful” (www.google.com/about/company). Fact is, the Web isn't organized by search engines at all. It's only indexed, which isn't the same thing.

See, the Web actually is organized, to some degree. That is, it has a directory, with paths to locations. Those paths can branch to infinity, but at least each one points somewhere. Search engines do crawl the Web's directory, and they do index its locations, but the index they search is a meta-version of the actual path-based Web. That meta-Web is organized by search engines into a giant haystack made entirely of needles. The job of the search engine is to make good guesses about the needles you might be looking for. At that they mostly succeed, which is why we have come to depend on them. But we're still dealing with a haystack of meta, rather than the real thing. This is important to bear in mind if we ever want to find a better way.

Lately Google has become convinced that the better way is to make searches more “social”. Its latest move in that direction is Search, plus Your World—a non-memorable name even when abbreviated to the unpronounceable SpYW (googleblog.blogspot.com/2012/01/search-plus-your-world.html). The “Your World” of which Google speaks is what you've organized mostly through Google + and Picasa, though it also indexes the “public-facing” stuff on sites such as Quora. But, if you're one of the 3/4 billion persons who live in the Matrix-like Facebook, SpYW won't help you, because Facebook won't let Google index your Facebook-based social stuff. Nor will it provide you with the option to expose it to SpYW or anything like it outside Facebook itself. Whatever the reasons for that, the effect is to partition the social Web into commercial zones with exclusivities that resemble AOL, CompuServe, Prodigy and other silo'd “on-line services” that the Web obsoleted back in the last millennium.

Why should we think that searches need to be social at all? There are lots of reasons, but the main one is advertising. Google, along with everybody else in the advertising business, wants to get personal with you. It wants to make the ads you see more relevant to you, personally. If it didn't want to do that, SpYW wouldn't exist.

Two problems here. One is that much of what we might search for (such as the case above) does not benefit from “social” help. In those cases, the social stuff is just more noise. The other is that our worlds, even on-line, are not just about stores, shopping and sites that live off advertising. They are much bigger than that.

In his new book, Too Big to Know: Rethinking Knowledge Now That the Facts Aren't the Facts, Experts Are Everywhere, and the Smartest Person in the Room Is the Room, David Weinberger writes, “Knowledge is now a property of the network, and the network embraces businesses, governments, media, museums, curated collections, and minds in communications.” Google still embraces all those things, but in ways that enlarge business at the expense of everything else. Google might not want to do that, but its advertising business can't help rewarding countless sites that live off on-line advertising, as well as businesses that advertise—while doing little for all the other stuff that comprises the Web.

I don't think the way to solve this is with a better search engine. I think we need something completely different—something that isn't so meta that we lose track of what the Web was in the first place, and why it is still an ideal way to keep and to find knowledge, and not just a place to shop.