UpFront

The kernel boot process is one of the strangest parts of the whole operating system. One aspect of it is initramfs, where the kernel must first boot up in a RAM-based root filesystem, before it can mount the disk-based filesystems and transition the kernel to consider those the root filesystem. initramfs traditionally has not been part of the official kernel, so each Linux distribution has come to maintain its own unique implementation. In general, this means the sequence of events in the boot process of each distribution is different. This can affect the way hardware is initialized and the state it's in when the user finally sees a login prompt.

Jeremy Katz recently announced Dracut, a new distribution-independent implementation of initramfs, inspired by discussions at the most recent Kernel Summit. The project's goal is to simplify and standardize the boot process and move the initramfs implementation into the official kernel tree. Each Linux distribution then would boot up identically, but would be able to layer any additional functionality on top of the main initramfs implementation.

Jeremy's work is based on the Red Hat initramfs. Hannes Reinecke is working on his own implementation, based on OpenSUSE. It's likely the two projects will merge at some point. The project will need to bear several issues in mind. As Neil Horman explained, some systems elect to stay in the RAM-based filesystem for stability reasons and never transition to a disk-based root filesystem. Any successful initramfs would have to support that feature, in order to work with various embedded system projects.

Keeping track of what goes into the kernel is always pretty tough. Each new version could contain thousands of patches. There are various ways to track them all, including using git or one of the git repository browsers. John Hawley recently set up a new method at patchwork.kernel.org. This page keeps track of all patches sent to a variety of kernel-related mailing lists, including the main one, linux-kernel. It presents the patches in color and is nicely formatted, and it lets folks change the status of each patch, so it's clear which have been accepted.

Although a variety of git documentation exists, some of it is fairly involved. For folks who'd rather just learn what they need to know, David Howells has written “The Git Hater's Guide To The Galaxy” and submitted it for inclusion in the kernel source tree. It's not the most in-depth git doc out there, but it covers a lot of the stuff David himself uses it for on a daily basis, and it may be more accessible than the more detailed docs out there.

One problem with running dæmons is that the init process loses track of the relationship between the dæmon process and the parent processes that spawned it. As Scott James Remnant has discovered, there doesn't seem to be any straightforward way in Linux for init to know by which process a given dæmon was spawned. That information is currently lost by the spawning procedure.

Scott has coded up support for some new signals to carry this information into the kernel. If accepted into the main tree, this code will allow system logs to track effectively the activities of the application that spawns any dæmons, along with the dæmons it spawns. Among other things, this will improve debugging efforts for any applications that spawn dæmons.

Most modern CPUs have special registers for tracking performance. Things like perfmon use these, but recently Thomas Gleixner and Ingo Molnar took a shot at designing something better. Their initial effort actually has met with fierce resistance from Paul Mackerras and David S. Miller, who felt that the approach Thomas and Ingo had taken was not likely to meet the needs of real systems. The debate does not seem anywhere near over, and certainly Thomas and Ingo will produce some code to prove their points. Probably something excellent will come out of all this, but currently, the two sides have not had much of a meeting of the minds. Paul has said there's no way to track a process effectively without stopping it, and Thomas and Ingo say, yes, it can be done. Time will tell.

From the Linux (and Linux Journal) perspective, there's an issue with clouds—those back-end Web services that compose Utility Computing. They're proprietary. Amazon owns AWS (Amazon Web Services: S3, EC2 and a growing number of others). Google, Microsoft and other companies own theirs as well.

These are open in the sense that most services are substitutable, which makes them utilities. If you don't like storing your data at Amazon's S3, you can park it elsewhere. The documents you create and keep at Google Docs can be put elsewhere as well. Elastic computing is more tricky, but it should be substitutable as well.

In a way, all these data centers are available to the public as a utility service. They are an important trend in computing, comprising the growth of a vast open SLA (service-level agreement) infrastructure for computing resources spread across the Net. (Nicholas G. Carr wrote about all this in his book The Big Switch—Rewiring the World, from Edison to Google. I visited the topic in “The Bigger Switch”, my EOF column in the May 2008 issue of Linux Journal.)

But, why should The Cloud consist only of proprietary back ends? How about building our own cloud services, our own way?

I was pleased to find an answer from colleagues at UCSB (where I'm a fellow at the Center for Information Technology & Society), namely Rich Wolski and his merry band of computer scientists in the Computer Science Department. There they have created EUCALYPTUS (Elastic Utility Computing Architecture for Linking Your Programs To Useful Systems). It's what they call “an open-source software infrastructure for implementing Elastic/Utility/Cloud computing using computing clusters and/or workstation farms”. It is interface-compatible with Amazon's EC2 (the de facto standard for elastic cloud computing) and designed for modification and extension as needed for you-name-it client-side interfaces. Most important for Linux Journal readers, “EUCALYPTUS is implemented using commonly available Linux tools and basic Web service technology making it easy to install and maintain”, they say.

The issue for IT folks is a range of choices that should include DIY as well as commercial back ends. Rich Wolski puts it this way:

As I write this, version 1.5 is coming along. (Version 1.0 was released in June 2008.) And, there are ambitious plans. For more information, check out eucalyptus.cs.ucsb.edu.

Freescale may be the first semiconductor company to associate itself aggressively with portable Linux devices. The former Motorola semiconductor division is sharply targeting the low-priced Linux-based Netbook market, which is hot in the world market and just starting to get warm in the US.

Early this year, Freescale launched a new processor called the i.MX515 Cortex A8 Netbook Processor. Company literature positions the part for “low-power, gigahertz performance Netbooks at sub-$200 price points”. According to CNET, Glen Burchers, director of global marketing for Freescale's consumer products group, says “Because the primary function (of a Netbook) is accessing the Internet, Linux and Firefox are a good operating system and application for that purpose.”

Freescale is developing a reference design with ASUS-subsidiary Pegatron. It will feature the 1GHz i.MX51 processor, Ubuntu Linux, Adobe's Flash Player, a new power management chip and the SGTL5000 ultra-low-power audio codec.

Some data:

The company is working first with Tier 1 OEMs (so watch for ASUS units) and expects products to start hitting the market in Q2 of this year—in other words, about now.

Resources:

Freescale Semiconductor: www.freescale.com/webapp/search/Serp.jsp

“New Freescale Processors Target Linux Netbooks”: arstechnica.com/journals/linux.ars/2009/01/07/new-freescale-processors-target-linux-netbooks

“Freescale Chip Aims at 1GHz, $199 Netbook”: news.cnet.com/8301-1001_3-10130690-92.html

“Freescale, Nvidia Chase Netbook Sockets”: www.eetimes.com/news/latest/showArticle.jhtml?articleID=212700361

1. Lowest legal price in dollars of the Ghosts I–IV album by Nine Inch Nails: 0

2. Price in dollars charged for Ghosts I–IV at Amazon.com: 5

3. Position of Ghosts I–IV among Amazon's top-selling albums for 2008: 1

4. Millions of female drivers in the UK who are “putting themselves and other drivers at risk by wearing the wrong footwear when behind the wheel”: 11.5

5. Preference percentage for a “higher resolution” photo that isn't, in a choice between two actually identical pictures: 75

6. Billions of dollars spent annually on Internet advertising by 2011: 106

7. Percentage of US Internet audience that viewed on-line video in November 2008: 77

8. Billions of on-line videos viewed by US Internet users: 12.7

9. Percentage increase of above over the same period in 2007: 34

10. Google's (Linux-served) YouTube rank as a US on-line video property: 1

11. Billions of YouTube videos viewed in November 2008: 5.1

12. Google Sites again ranked the top US video property, with nearly 5.1 billion videos.

13. YouTube percentage of all videos viewed in November 2008: 40

14. YouTube share of Google video properties: 98

15. Millions of US Internet viewers in November 2008: 147

16. Average number of videos users watched by US viewers in November 2008: 87

17. Duration in minutes of the average on-line video: 3.1

18. Duration in minutes of the average on-line video viewed at (Linux-served, via Akamai) Hulu.com: 11.9

19. Number of results in a search for “Linux” at AMD.com (US): 1,281

20. Number of results in a search for “Linux” at Freescale.com: 1,445

1: Nine Inch Nails

2, 3: Amazon.com

4: Sheila's Wheels (a UK insurance company), citing a 2007 YouGov survey of 754 female drivers

5: ArsTechnica, citing Journal of Consumer Research

6: CRM Today

7–9 and 11–17: MarketingVOX, sourcing comScore's Video Metrix, Marketing Charts

10, 18: SearchDNS.Netcraft.com

19: AMD

20: Freescale

That's what it's called. I found out about it when a pal sent a link under the subject line “Hell hath frozen over.” Look up the above on Google and you'll find it. Blogspot, Netcraft tells us, is in the Google, Inc. netblock, and running on Linux.

I was pointed recently to Zappos (www.zappos.com) as a near-perfect example of a company that brings the principles of open source to business. Its site is inventive and fancy (as you'd expect a clothing retailer to be), but not a triumph of design over utility. What's more, it's fast. I can check Web site responsiveness with some confidence, because our little apartment near Boston has 20Mb/s symmetrical service from Verizon FiOS (that's fiber, and pretty cheap, considering), and the speeds I get at the office I share at Harvard are more than twice that.

So I decided to see what they were running that site on, by checking with Netcraft.com's “What's that site running?” service. The answer was Linux. Zappos itself is in an Akamai Netblock, and of the 13 other results (all foo.zappos.com), the results for OS were Linux or “unknown”.

Then I decided to look beyond Zappos to other on-line retailers. Here are the results for the top ten, as listed by InternetRetailer.com, with sales volume numbers in parentheses. Results for companyname.com are first, and other results for each company are summarized in text (Netcraft gives results in lowercase):

Although that summation is far from a complete picture, or even a completely accurate one for this Top Ten, it's clear that the picture can't be painted without linux as the most primary color.

Resources:

Internet Retailer Top 500 Retail Web Sites: www.internetretailer.com/top500/list.asp

Netcraft: netcraft.com

If computers of the kind I have advocated become the computers of the future, then computation may someday be organized as a public utility. We can envisage computing service companies whose subscribers are connected to them by telephone lines....

The system could develop commercially in fairly interesting ways. Certain subscribers might offer services to other subscribers. One example is weather prediction. A weather-predicting company that is a subscriber to the central computer predicts the weather but keeps the predictions in its private files. If you subscribe to its service, your programs can gain access to those files. You may even have weather-predicting programs run for your benefit to answer your own particular questions....Other subscribers might furnish economic predictions. The computing utility could become the basis for a new and important industry.

—John McCarthy, in 1961

At present, a newspaper, magazine or book is a package produced by a large organization. With the information utility, the physical production and distribution disappears, allowing a much smaller organization to put out the same packages of text and pictures. Moreover, the user does not face a one-shot decision to buy Time or Newsweek. He will be able to read the “cover” or table of contents of each, read such items as strike his fancy, and the system will bill him for what he reads from each source. In fact, since the cost of keeping a file of information in the computer and making it publicly available will be small, even a high school student could compete with the New Yorker if he could write well enough and if word of mouth and mention by reviewers brought him to public attention.

—John McCarthy in the 1970s, www.theregister.co.uk/2005/11/10/gates_ozzie_memos/print.html and www-formal.stanford.edu/jmc/future/hoter1.html

Does anyone really think that a company-owned platform is going to win here, that it won't be swamped by an open federated system of servers that peer, like e-mail? If so, I'd like to hear why. We went through this exercise repeatedly in the tech industry; the lesson of history is clear—closed systems have their place and time, at the beginning of a new layer, when users need simplicity over everything else, they serve as training wheels when everyone is a newbie. Eventually we grow out of the need to have our hands held and the freedom of open systems becomes attractive, and we jump.

—Dave Winer, www.scripting.com/stories/2009/01/03/helpingFriendfeed.html

Nobody will doubt the value of openness for the mobile industry anymore.

—Sean Moss-Pultz, article.gmane.org/gmane.comp.handhelds.openmoko.community/33625

as a thank you to our fans for your continued support, we are giving away the new nine inch nails album one hundred percent free, exclusively via nin.com.

the music is available in a variety of formats including high-quality MP3, FLAC or M4A lossless at CD quality and even higher-than-CD quality 24/96 WAVE. your link will include all options—all free. all downloads include a PDF with artwork and credits.

for those of you interested in physical products, fear not. we plan to make a version of this release available on CD and vinyl in july. details coming soon.

—Nine Inch Nails, dl.nin.com/theslip

While Linux-based netbooks have not gained much consumer acceptance in the US, their success varies greatly by geographical market. In developing countries, Linux-based Eee PCs have fared better....Even if a majority of netbooks run Windows, the minority that run Linux are the most successful non-Windows, non-Macintosh consumer PCs in the industry in terms of penetration.

—Forrester Analyst J P Gownder, www.vnunet.com/vnunet/news/2233381/netbooks-sales-driving-linux

We love Web videos. We especially love videos that teach us new quick-and-easy tricks, so we've been cooking up some more videos over in the LinuxJournal.com kitchen. We'll have several short video tips per week in the Tech Tip of the Day section. Visit us often to see some fun tricks to make your life just a little better. Here's one of my favorites: www.linuxjournal.com/video/transfer-your-terminal-screen.

I first met Phil Jacob on a consulting gig (yes, I got paid, but that was a long time ago), and quickly discovered that he was one of our most careful readers. Subsequently, he also became one of our most constructive critics—in the best sense of the word.

Phil is a born connoisseur. He applies that nature as founder and CTO of StyleFeeder (www.stylefeeder.com), a startup with a “personal shopping engine” that combines machine learning techniques and social data to provide users with a richer shopping experience. Among the features under StyleFeeder's hood is one that personalizes search results from a dataset of more than ten million products using a custom-built recommendation engine. It also has a mature data tier shared across 100 MySQL databases in order to handle its scalability requirements. Given that StyleFeeder is only six people, they've done a lot with a little during the past two years. Phil also is a technology advisor to PRX, a nonprofit licensing and distribution platform for public radio content. He's also married to a well-known food blogger, Beatrice Peltre, who runs La Tartine Gourmande (www.latartinegourmande.com), so at least he eats well.

When I asked him to share some of what he's using, he concentrated his report on a single application—music:

Asked for last words on the subject, Phil adds, “We run RHEL and CentOS at StyleFeeder, some on EC2 and some at Contegix in St. Louis. We run behind Apache and nginx on the Web tier. We own zero servers and run dd-wrt on our office LAN as well....Typing one-handed with a baby in the other....” Babies are a new hack for Phil.