UpFRONT

Doc Searls

Issue #82, February 2001

Stop the Presses, LJ Index and more.

FAST Times

A year ago, not many people knew about Google. Being the search engine both by and for the Linux community, now everybody knows about Google. It doesn't hurt that Google now powers Yahoo's Web searches as well as its own.

While Google remains a fine search engine, the company's decision to patent its search methods hasn't sat well with those in the Linux community who don't cotton to software patents, which include many in the Free and Open Source Software communities.

Well, there are also other search engines with Linux and UNIX credentials. One is Fast Search and Transfer ASA (FAST) http://www.alltheweb.com/, a Norwegian company with offices in the US and a partnership with Dell. The two companies jointly and publicly intend to build the world's largest and deepest search engine.

Early last year, Lycos made a substantial investment in FAST and now co-brands FAST's four basic search engines: FAST Web Search, FAST FTP Search, FAST MP3 Search and FAST MultiMedia Search (all of which can be found at www.alltheweb.com and www.lycos.com/—they use the same engines).

FAST's engines run on FreeBSD and are reportedly developed on a mix of FreeBSD and Linux machines. In fact, FAST's first engine, FTPsearch, was developed under the Free Software Foundation's GPL. You can still download the GPL version of that software at ftp://ftpsearch.ntnu.no/pub/ftpsearch/. Search results are presented by Apache and PHP.

We also understand that some of FAST's people have been involved in PHP's development for a long time, and many of FAST's R&D people in Norway come from one UNIX-oriented computer club at the University in Trodheim. It's called “Programvareverkstedet”, or PVV http://www.pvv.org/.

The products FAST sells are closed-source along with the search engine itself, which is also the case for every other search engine at this point (or at least that we know of—correct us if we're wrong).

For more about FAST's technologies, click the “Technology” tab on the company's home page.

In another significant search engine development, Yahoo began in November to charge businesses to hurry their listings into Yahoo's “Business to Business” and “Shopping and Services” areas within the “Business and Economy” category. For $199, Yahoo's Business Express program fast-tracks submissions for review and possible inclusion in Yahoo's listings in either of those two areas. According to the FAQ docs.yahoo.com/info/suggest/faq.html, “...any site submitted to these areas will be reviewed and either added or denied within seven business days. If your site is denied, you will be told why and will have a chance to appeal the decision.”

Meanwhile, the Open Directory Project (http://www.dmoz.org/) continues to grow at an explosive rate. A cursory set of searches shows the two services are highly competitive. The question now is, how do they scale?

There's not much you can do to help Yahoo other than work for the company or pay for a listing. But there's a lot you can do to help the Open Directory Project—mainly as an editor. Just navigate down to a topic that obsesses you and sign up to become an editor through the link on that page.

Copywright, Guthrie Style

When Woody Guthrie was singing hillbilly songs on a little Los Angeles radio station in the late 1930s, he used to mail out a small mimeographed songbook to listeners who wanted the words to his songs. On the bottom of one page appeared the following: “This song is Copyrighted in U.S., under Seal of Copyright # 154085, for a period of 28 years, and anybody caught singin it without our permission, will be mighty good friends of ourn, cause we don't give a dern. Publish it. Write it. Sing it. Swing to it. Yodel it. We wrote it, that's all we wanted to do.” —Pete Seeger, June 1967

LJ History

In the February 1995 issue of Linux Journal Belinda Frazier reports on Comdex 1995 and its Linux presence:

...there usually isn't much about UNIX at Comdex. This year however, I was very pleased to find Linux represented at two booths at the show. Both Yggdrasil Computing, Inc. and Morse Telecommunication had Linux in their companies' banner.

In comparison, at the Fall Comdex, the number of exhibitors in the Linux Business Expo section was in the neighborhood of 500.

LJ INDEX—February 2001

  1. Lines of code in the average electronic toothbrush: 3,000

  2. Millions of billions of calculations per second in the human brain: 20

  3. Billions of embedded processors sold in 1998: 4.8

  4. Percentage of embedded processors intended for PCs in 1998: 2.5

  5. Number of embedded chips in the typical family car: 20

  6. Number of microprocessors in the typical American household: 40

  7. Trillions of dollars in projected on-line business-to-business sales in 2002: 1.3

  8. Increase in Americans' exposure to advertising between 1971 and 1991: 6x

  9. Percentage of stress-related visits to physicians: 75 to 90

  10. Increase in meat consumption during the last population doubling: 4x

  11. Species of diseases: 250

  12. Species of weeds: 220

  13. Percentage of grain fed to livestock: 40

  14. Number of trees it takes to print the Sunday New York Times: 3,200

  15. Millions of prisoners in the US in 2000: 1.3

  16. Percentage of Americans who will spend time in prison, if current incarceration rates continue: 5

  17. Percent chance for an African-American male in the US of going to jail: 28

Source:

Linux Bytes Other Businesses

by Heather Mead

Prior to what Omaha Steaks (http://www/omahasteaks.com/) believed would be the start of their busiest on-line shopping time of year, the 2000 holiday season, they decided to revamp their web site to accommodate an estimated one million customers. eOne Group (http://www.eonegroup.com/), an Omaha-based eCommerce software and service provider, worked with Omaha Steak's internal IT department to build a web site that could be hosted on-site, integrated into back-end systems as well as provide advance delivery schedules and multiple ship-to-address options.

As the basis of the new web site, eOne used their Java-based, hardware-independent development application called jCommerce. Being an open-architecture application, jCommerce runs on almost any platform and can be customized for each individual customer and situation. As jCommerce provided the database-driven features Omaha Steaks wanted, including easily customized XML and XHTML tags and accessible userinterfaces, the software choice seemed easy enough; the real decision centered on what platform to use.

Chad Bukowski, chief architect at eOne, characterizes Omaha Steaks as a traditional, family-owned and operated business since 1917. He was unsure how receptive the company would be of the open-source philosophy and business model Linux offered, the platform eOne actively supports for their jCommerce software. But he also knew that their final decision would be based on price and performance and not necessarily the name attached to the platform. The benchmark stress and load tests were done on RS/6000 M80 IBM systems, their current AS/400 system and Dell Intel boxes running Red Hat Linux 6.2. During the test, the AS/400 system slowed when the number of separate, simultaneous customer orders headed toward 50; the RS/6000 topped out around 150-200 orders. The Dell boxes, according to Bukowski, processed 250-400 orders with little taxation.

Impressed by the benchmark results of Linux, its scalability and because Omaha Steaks “wanted something that wasn't tied to IBM”, Bukowski said that they were eager to launch their new web site running on Linux. Jeff Carter, CTO of Omaha Steaks, said that four main considerations made Linux the most viable option: price performance of the Dell servers versus the IBM offerings ($8,000 per machine for Dell vs. $250,000 per machine for RS/6000, plus licensing fees); overall acceptance and performance of Linux in the Internet space; stability of Linux versus NT on the Intel platform; and easy integration of Linux with legacy systems.

The new site went live in fall 2000, after a 60-day setup period. So far, both Omaha Steaks and eOne are pleased with the results. Reboots are nonexistent, and Bukowski says the system “is singing right along”. Customers are satisfied with the simplified and versatile ordering procedure, and Omaha Steak employees have had no problems adapting to the new system. Carter underlines the importance of having a system that can be maintained with internal resources: no Java programming is necessary to maintain the parameter-driven site. Of Omaha Steak's relationship with eOne Group, jCommerce and Linux, Carter says, “No one else in the marketplace had all of these thing to offer.”

Apache Extends the Plateau

Netcraft's monthly survey (www.netcraft.com) of hosts providing HTTP service has told the story of Apache's long-running leadership ever since the survey began in mid-1995 when Apache weighed in with a share of just a few percents. Late that year, Apache, NCSA and “other” were all tied at around 30%. This was also when Microsoft's IIS came on the scene. Since then Apache and IIS have gained and held the leading two positions, with Apache taking the majority share by a wide margin. While IIS overtook “other” to assume second place in December 1997, with about 20% of all sites surveyed, Apache was already moving past a 50% share.

The story told since late 1998 has been a pretty stable one. Apache holds a strong 59.67% (down from 60.02%). Microsoft is up from 19.56% to 20.17%. Sun's iPlanet (which includes all the old Netscape servers) is down from 7.15% to 6.92%. But those are just shares of a total that is up for all three. Here are the gains:

  • Apache: up 590,305—from 12,705,194 to 13,295,499

  • Microsoft: up 352,960—from 4,140,977 to 4,493,937

  • iPlanet: up 27,169—from 1,514,106 to 1,541,275

The rising tide lifts even the most competitive boats.

The most interesting news coming out of Netcraft's latest survey concerns uptime. “Hosting locations with at least five popular sites queried several times each week are reported in order of average uptime, and a league table of the fifty sites with the highest uptimes on the Internet is also maintained”, Netcraft reports. The resulting graphs are pretty interesting. We see that Slashdot was rebooting almost daily going into late 1999 when uptime began to move past 60 days. One server has been up since May of this year (we're writing in November). Slashdot right now has a very respectable maximum uptime of almost 190 days and a 90-day moving average of over 120 days. Netcraft reports that Slashdot is running “Apache/1.3.12 (UNIX) mod_perl/1.24 on Linux”. By contrast, Microsoft has a max of 75.22 days and a 90-day moving average of 18.76, running “Microsoft-IIS/5.0 on Windows 2000”. To be fair, however, we can point to Starbucks, which now is past 215 days of uptime since moving to Windows 2000. This is also “the longest uptime of any Windows 2000 site we've seen on the Internet”. The overall record-holder is www.charite.de, which has been up 836.49 days or more than 2.5 years. It's running “Netscape-Commerce/1.1 on IRIX”.

By far the largest hosting site is Exodus Communications, which has 548 sites at just one “netblock” in Santa Clara, California. The four longest-running servers are all for Zope and all run Linux. They're averaging an uptime of 384 days, which also happens to be the entire measured period. Three run “Zope/(unreleased version) ZServer/1.1b1”. The other runs “Apache/1.3.4 (UNIX)”. Also on the Exodus list are smellygig.com (Apache on Solaris, 277 days), cluetrain.com (Apache on Solaris, 258 days), mysql.com (Apache on Linux, 183 days), whitehouse.com (yes, the XXX site, Apache on BSD, 162 days) and slashcode.com (Apache on Linux, 190 days).

Another interesting Netcraft observation concerns “how good the vendors are at running their own kit”. Among the list of hosting locations with the longest uptimes, VA Linux is in sixth place. Sun is at ninth. Microsoft is at 25th.

Netcraft's latest survey information is available at http://www.netcraft.com/survey/.

Say Where?

Disturbing Search Requests (DSR, http://www.searchrequests.weblogs.com/) is a “collaborative weblog dedicated to misleading search requests” that mines the veins of irony found in both site referrer logs and search-engine results derived from link-loving bots and engine algorithms that give maximum value to the most highly linked sites. So, DSR continues,

If you write a weblog on a regular basis, chances are you're going to post quotes from other sites, opinions from other people, etc. But since weblogs are highly linked to and from, they get indexed very well by search engines. So, even if you only once wrote about your hamster, and on the same day mentioned you were wearing a three-piece suit, Google just might list you as No. 1 for “hamster suit”. Now just imagine that you check your referrer logs and you find a query from a search engine, looking for “hamster suit”. This is where this site kicks in.

For example, Dave Bug of Geeklife (http://www.geeklife.com/) found that “Where are all the nice girls?” was not only a search request in his log files, but brought up The Value of Psychotic Experience (deoxy.org/w_value.htm) as a result on Google.

At this writing (November 20), there seems to be agreement among threads both within and linked to DSR, that Google and Yahoo (which uses the Google engine) sometimes exclude DSR from its findings. So we just typed “Disturbing Search Results” into Google, punched the “I'm feeling lucky” button and went straight to the site. Guess this isn't one of those times.

A New Linux Site for the Spanish-Speaking Linux Community.

Piensa.Com Systems has launched La Gaceta de Linux at http://www.gacetadelinux.com/. This site is the host of the Spanish edition of our sister on-line publication Linux Gazette. Gaceta de Linux is focused on providing firsthand information to the global Spanish-speaking Linux community. The site is made possible by the efforts of bilingual, English-Spanish volunteer translators who choose and select original articles in English and create a high-quality translation in just a few days. “We are fully committed to the Latin-American and Spanish-speaking use of Linux, especially in real-life business applications,” commented Felipe Barousse, CEO and General Director of BCM Piensa.Com Systems. “We believe Linux gives us a tremendous opportunity to spread technology with minimal cost and extremely high reliability and success where it is not feasible to deploy other proprietary options. La Gaceta de Linux is a great vehicle for this endeavor. Also, by the time you read this, the Portuguese edition, called Gazeta do Linux should be on-line as well.”

Translations are made by volunteers, and all articles in fact, are reviewed for quality and content, always respecting the original author copyrights. They have registered users from more than 22 countries and more than 100 translated articles since Gaceta's opening last June. To join, log onto www.gacetadelinux.com/register and help spread Linux in the Spanish and Portuguese-speaking regions of the world.

Copyright, Guthrie Style

Stop the Presses: the Shift from Why to Why Not

At Comdex Fall in Las Vegas, the North Hall was where the wireless and telecommunications companies were concentrated. In the midst of the Internet Appliance pavilion was a booth for Be, Inc., the operating system company that has recently retooled itself to go after the Internet appliance business. While Be was making a valiant effort, the big news was quietly evident in other booths all around the floor, where Linux had quietly soaked into everything.

A company called Internet Appliance showed racks of servers featuring advanced failover technologies, dynamic content screening, browser-based setup and administration and other goodies one would hope to find in “a complete Data Center for less than $30,000”. Does it run on Linux? Yes. Is that fact a big deal? No.

While Be hustled for every bit of visibility it could muster, Linux was quietly demonstrating its new default status as commodity OS infrastructure. Here are just a few of the Linux-powered devices I encountered at the show:

  • Sony VAIO C1 PictureBook notebook computer (running on a Transmeta Crusoe processor)

  • IBM NetVista N2200 thin client

  • Ericsson Nanorouter 2 communication gateway and server

  • Nokia Mediascreen Multimedia terminals

  • e-Appliance SuperScaler network appliances

  • Agenda VR3 sub-Palm-size PDA

  • Snap Servers—portable snap-together servers from Quantum

  • ZFLinux Devices—Linux embedded on an X86 chip (and chip set)

  • Gateway's Connected Touch Pad internet appliance (also with a Transmeta Crusoe), which won “Best Consumer Product” from ZDNet and CNet judges

When I spoke with these companies in their booths, most didn't even give me reasons why they built their boxes with Linux. It was as if I asked why they used a TCP/IP stack or plastic in their cases.

Embedded Linux has become viral. Even when it isn't the subject at hand, the fact of Linux's rapid spread into embedded devices still comes up. While I was talking with a couple of guys at the Micronas booth (Micronas is a German company whose multimedia chips are in pretty much everything), they told me about a company called NetGem that makes Linux-based appliances that use televisions as monitors—very big in Europe, they report.

No doubt Be has a lot to offer. But it has to be sold. At this point, embedding Linux isn't even much of a brainer anymore. At some point in recent history, the question shifted from “why” to “why not?” Clearly there are fewer reasons every day.

They Said It

The seven worst words in cyberspace are “You just don't get it, do you?”

—Bob Metcalfe

There's no Moore's Law for software.

—Ellen Ullman

Like constipated feces, on-line advertising giant DoubleClick Inc. and handheld Internet-content leader OmniSky Corp. have impacted themselves within the growing intestinal bolus of companies experimenting with sending ads to wireless gizmos.

—Tom Matrullo

We've embarked on the beginning of the last days of the age of oil.

—Mike Bowlin, Chairman/CEO, ARCO

Build-to-flip financing is not a business model, it is a risk cap model.

—Patrick Sweet

Every extension is an amputation.

—Marshall McLuhan

Talent is always conscious of its own abundance and does not object to sharing.

—A. Solzhenitsyn

If the automobile industry had made as much progress (as the computer industry) in the past fifty years, a car today would cost a hundredth of a cent and go faster than the speed of light.

—Ray Kurzweil

Linux has to be important to General Motors. Not just to gm.com.

—Larry Augustin, VA Linux

Where there's chaos there's opportunity.

—Bryan Sparks, Lineo

Without chaos there would be no fun and games.

—Jonah Kinata

Linux is quickly becoming the operating system of choice for any new device, and the silicon vendors are sponsoring that.

—Inder Singh, LynuxWorks

There is an explosion of new devices, and more and more the assumption is that new devices will communicate.

—Michael Tiemann, Red Hat

Where devices are connected, Linux shines.

—Bryan Sparks, Lineo

The cost of hardware, software and network bandwidth are going to zero. When all three go to zero, the embedded space becomes infinite. So there is no reason anything you own should not contain some kind of intelligence. Linux as software is already there.

—Michael Tiemann, Red Hat

Tech Tips from Our Tech Editor

If you're running IP Masquerading and want to make services on your internal network available from the outside, just install rinetd on the IP Masquerding box available at http://www.boutell.com/rinetd/. (Also available as a package; check your distribution's web site.)

If you let other people run Perl scripts on your machine, make a symlink from /usr/local/bin/perl to the real location of Perl so they don't complain, “Hey, you don't have Perl.”

Another thing that shouldn't make any difference but sometimes fixes weird problems: switch PCI cards around in their slots.

If you have a complicated crontab, do a crontab -l > my_crontab_`date +%Y%m%d' to save it in case you royally mess up a crontab -e.

vi Navigation

Use (, ), [, ], { and } for navigation marks in vi. The % command will move the cursor to the matching mark. For example, with

( some text [ some more stuff
another line
{  ]
} )

If you position the vi cursor on any of the marks, in command mode the % will move the cursor to the matching mark.

You can use this method in comments in a programming or scripting language to allow quick fast-forward and fast-backward over large blocks of code, as well as use it to find boundaries of functions in languages like C which use one of these marks to delimit functions.

Mapping

Map an unused key to change files in vi. For example, if your .exrc or .vimrc has map , :e#^M when you've edited two files, for example, by giving

vi file1 file2<Enter>
:n<Enter>

so that you're in file2 and in command mode, typing , (no Enter required) will switch you back to file1. Use it again, and you switch to file2 and so on.

Job Opening Trends

by Reginald Charney

The falling fortunes of the dot-coms and the uncertainty of the election results, even before the election, have affected job figures. While the number of jobs offered has fallen since April, the decline has leveled off. Chart #1 shows the normalized job trends over the last 11 months. Ordinarily, things that don't change are not very interesting. But in this case, the fact that things have leveled off is good news.

(Note: Chart #1 is normalized for the number of jobs in January of this year. That is, the number of openings in January 2000 has been taken as 1.00.)

Figure #1

Platform Demand

Over the last 18 months, a number of platforms have competed for dominance. One of the interesting aspects of this is how fast demand for the main platforms has been growing or shrinking. Demand is defined as the rate of acceleration/deceleration in the trend line for the period shown. From Chart #2, we can see that demand has been slowing for the older platforms, while the newer ones, like Windows 2000 and Linux, have accelerated faster than all others.

Figure #2

Again, the exception to the rule is Solaris. It is right up there with the newbies. This indicates that Sun is increasing its dominance of its markets. These demand lines are positive at the moment because of the long period of expansion. Over the last 30 days, demand for all platforms has followed the general trend and decelerated. However, the short term has not yet outweighed the long-term demand.