UpFront

diff -u: What's New in Kernel Development

Zack Brown

Issue #147, July 2006

Paul Mundt has submitted patches to remove RelayFS from the kernel and migrate all of its functionality into a generic API that can be used by any filesystem. Thus, what started off as a highly specialized tool—a filesystem only for high-speed data transfer between user space and the kernel—has now been generalized into a service with wide-ranging applicability. Because RelayFS has been in the official kernel for some time, taking it out now already has produced some controversy. But, apparently, Andrew Morton promised the RelayFS developers that they could continue to make these large, sweeping developments, so long as no in-kernel code relied on RelayFS. User applications should not have been relying on RelayFS during this time of instability.

Intel has announced a new open-source project to support its PRO/Wireless 3945ABG Network Connection mini-PCI express adapter (IPW3945). It is not fully open source. There is a single binary-only part, containing the regulatory enforcement logic for all the countries where those adapters ship. In his announcement, James Ketrenos said that Intel had improved its licensing relative to earlier projects. The binary portion of this project uses the same license as the binary-only firmware, and James has said it is easier to understand and more permissive of redistribution than earlier licenses. The whole issue of regulatory enforcement and binary distribution is, as one might expect, controversial. The binary dæmon must run as root, which makes any potential bugs a large security problem. Also, there is some question as to whether the FCC regulations actually require binary-only distribution, or only, as Alan Cox puts it, “that the transmitting device must be reasonably tamper-proof”. Regardless of any controversy (that is sure to be ongoing), Intel can at least be credited with freeing the portions of its code it has freed and with making some effort to improve its license.

Bernhard Rosenkraenzer has forked the cdrtools project into his own dvdrecord project, and released version 0.3.1 with various enhancements, such as supporting 2.6 kernels, supporting writing to DVD-R and DVD-RW disks using purely free software and cleaning up the make system. Apparently, there have been massive flame wars precipitating the fork, and a number of kernel folks have said that Bernhard's work represents the only progress on the driver in recent days. The conversation has turned toward whether his dvdrecord will support additional features, such as DVD RAM, DVD+R, DVD+RWand DVD+DL. It's unclear what the long-term direction of the project will be, because forks are always controversial, but at least there seems to be no immediate outcry that Bernhard forked the project unreasonably. It's still too soon for predictions, however.

Miklos Szeredi has created the mountlo utility, a tool that supports loopback mounting in user space. Until now, a filesystem image stored in a single file on disk could be mounted only via the kernel's own loopback support. Miklos' utility relies on FUSE (Filesystem in USErspace) to move the entire feature into userland and out of the kernel. Miklos considers it more of a pet project than anything else—an opportunity to play with FUSE and see what useful tools he could create.

The ever-vigilant Christoph Hellwig has pointed out that the gdth driver seems to be the only user of the scsi_request interface expected to be gone in Linux 2.6.17. The gdth maintainers have not responded to patch submissions, and unless someone steps forward to make sure gdth survives the interface change, it will be marked as “BROKEN” in 2.6.17. Achim Leubner has stepped forward to do some testing, but this leaves the question of driver maintainership up in the air. Typically, unmaintained code is rapidly deprecated in Linux.

Greg Kroah-Hartman has begun to document ABI (Application Binary Interface) levels of stability within the kernel. When a binary interface changes, user-space binaries that link to that interface break. And, because a given interface may have countless applications relying on it, it is considered virtually unacceptable to change the kernel ABI for that reason. Unfortunately, ABI changes are a fact of life. They do change, and have been changing, and the question as Greg sees it, is how to balance this with the needs of userland. Greg's idea is to provide enough indication about the future of a given interface that application developers will have time to rewrite their code before the change takes place. As one might expect, this is an incredibly controversial issue. Many top kernel people feel that the ABI is sacrosanct and should never be altered. But Linus Torvalds, in spite of having criticisms of the specific details of Greg's effort, seems generally to agree that ABI changes are inevitable, and that the kernel should do what it can to ease the burden placed on application developers.

LJ Index, July 2006

Doc Searls

Issue #147, July 2006

1. Minimum bandwidth in Mbps required for 1080p HDTV: 30

2. Days required to download a standard 7.8GB DVD movie file over 56Kbps dial-up: 13

3. Record number of terabits per second sent over a 160km link: 2.56

4. Equivalent number of DVDs per second: 60

5. Percentage of San Francisco Mayor Gavin Newsome wants covered by free Wi-Fi: 100

6. Percentage of on-line consumers who express an interest in podcasts: 25

7. Percentage of on-line households in North America that regularly download and listen to podcasts: 1

8. Versions of UNIX running at Microsoft's Open Source Software Lab: 15

9. Versions of Linux running at Microsoft's Open Source Software Lab: 50

10. Billions of dollars in PC-based point-of-sale (POS) systems sold in North America in 2005: 6

11. Linux percentage share of North American POS system sales in 2005: 9

12. Linux percentage share of North American POS system sales in 2004: 5

13. Millions of dollars in Linux sales in China in 2005: 11.8

14. Percentage growth of Linux in China from 2004–2005: 27.1

15. Expected compound annual growth percentage rate of Linux sales in China over the next four years: 34

16. Projected millions of Linux sales, in dollars, at the end of the next four years in China: 51.1

17. Percentage probability that open source will compete with closed-source products in all infrastructure markets by 2008: 80

18. Percentage probability that 75% of all mainstream IT organizations will have formal open-source acquisition and management strategies: 80

19. Percentage probability that mainstream IT organizations will consider open-source software in 80% of their infrastructure-based software investments: 70

20. Percentage probability that open source will be included in mission-critical software portfolios within 75% of Global 2000 enterprises: 90

1: Wayne Caswell

2–4: Gizmag

5: New Media Musings

6, 7: Forrester Research

8, 9: Microsoft Open Source Software Lab

10–12: LinuxDevices.com

11–16: Silicon.com, sourcing International Data Corp. (IDC)

17–20: Gartner;

Signs of GhandiCon 4

Doc Searls

Issue #147, July 2006

“First they ignore you, then they ridicule you, then they fight you, then you win.”—Mahatma Gandhi

For many years, Linux and open source were synonymous with “threat” at Microsoft. Although that status may not have changed, the company has been moving gradually in a more accepting direction toward market opponents that include, in some cases, its own customers.

We've seen acceptance, for example, in the Identity Metasystem, which was a topic of my Linux Journal column in October 2005, and also with the company's embrace of blogging—a product of open standards and practices that grew up outside Microsoft's (or any vendor's) platform.

But with Microsoft's Open Source Software Lab, which started up in August 2005, acceptance was elevated above the purely de facto level. It became policy. Then, in April 2006, Microsoft opened the lab itself, through a new Web site called Port 25. Named for the router port for outbound e-mail, the site is open, interactive and intended to foster helpful communications with Microsoft customers that use open-source software as well.

Although this hardly means Microsoft will be developing open-source software any time soon (especially for Linux), it may help ease tensions in the heterogeneous environments where customers require productivity, and not merely coexistence.

I asked David Sifry, founder and CEO of Technorati (which runs on Linux), for some perspective on what it means. He said, “It's good to see Microsoft recognizing the impact and importance of open-source software, and I'm encouraged by its recent moves to understand and work with the large community of nonproprietary software developers out in the world. In the end, it will be good for Microsoft customers.”

www.forbes.com/home/enterprisetech/2006/03/22/ballmer-microsoft-linux-cz_df_0322microsoft.html

RubyGarden

Visit www.rubygarden.org for all your Ruby needs, including links to other all-important Ruby sites, such as Pragmatic Programmers and RubyForge, as well as links to Ruby Webloggers.

RubyGarden provides an exhaustive listing of Ruby User Groups around the world at www.rubygarden.org/ruby?RubyUserGroups.

RubyGarden also recommends other sites of interest for Ruby User Groups, including ruby.meetup.com, where you can find a meetup group near you, and www.rubyholic.com, where Ruby groups can register and post meeting information.

You also can join the Google Ruby Brigade Group, at groups.google.com/group/Ruby-Brigades or by sending an e-mail to Ruby-Brigades-subscribe@googlegroups.com.

Tabblo: Pictures of Linux at Work

Doc Searls

Issue #147, July 2006

If you're looking to make something more of your digital photos than the usual on-line gallery, Tabblo has ways to do it. In addition to your basic GIMP (or, to the less free, Photoshop) tools (crop, zoom, add text and so on), Tabblo lets you lay out, annotate and share “tabblos” (a pun off “tableaus”) of photo sets, or output them in various print forms through off-line services—all from inside a browser.

Tabblo Creator Antonio Rodriguez and Python Creator Guido van Rossum

As Tabblo founder and CEO Antonio Rodriguez puts it, “If you have a story to tell with pictures, we have ways for you to tell it—on-line and off.” He also says Tabblo's own story would not be possible without Linux. To get that story, we did a brief interview:

LJ: Why this business, and why now?

AR: This is something the market has needed for a long time, but the conditions weren't right. The Web browser wasn't built to do serious multimedia processing, and until recently, it wasn't possible to do it on the server—at least not with economics that worked for anyone trying to build a business out of it. If you wanted to do server-side image editing at high scale, for example, the cost of the software and hardware involved was simply prohibitive. If you wanted to have a rich database of things like metadata-based navigation and unlimited undo, the cost of the Oracle licenses at volume would probably put you flat out of the business of providing any meaningful functionality for free in order to attract customers. And most important, if you wanted to store gigs and gigs of people's pictures, NetApp or EMC were the only games in town—at economics that make sense only for investment banks and NASA. I can say without any doubt that there was absolutely no way that we could provide the level of application given to users at launch for free without the benefit of our modified LAMP stack and its associated development practices. I am particularly sensitive to this because of the number of photo sites in the past that have actually tried to make money and have been undone by spiraling capital expenses and wrong assumptions about the likely uptake of revenue-generating products.

LJ: What's in your stack?

AR: For hardware, we use custom-made whitebox AMD64 boxes with plenty of horsepower per dollar of cost, running Debian AMD64 (which greatly lowers our sysadmin costs). We run our own clustered filesystem built mostly on top of the Apache 2.0 stack (as modules) and get throughput that commercial storage solutions can't touch, mostly because the software is written specifically for our workload. The image servers also leverage the Apache runtime, as well as ImageMagick. Our database is MySQL5, which is great for its ability to be set up in very funky configurations that let you optimize writes, reads or replication depending on your own needs. And, finally, our Web app is written in Python, which is both lightning fast to develop in and incredibly clean as a dynamically typed language.

LJ: How do you plan to evolve the service?

AR: To start, we're going for composition, layout and effects. Next is printing and distribution. We also plan to make everything publishable to blogs though the metaweblog API.

Tabblo also makes the most of what's “mashable” from other Web services, over open APIs. When Antonio showed me Tabblo's beta service at O'Reilly's eTech Conference in February 2006, he populated my own Tabblo library by copying more than 6,000 photos from my collection at Flickr. He did this also to demonstrate the well-behaved nature of Flickr as a Net citizen. Rather than locking up customer data, Flickr is wide open, allowing users on other services to access and use photos, including all kinds of photo metadata, as well as tags. Tabblo, he told me, also aims to be equally responsible to what he calls “the open marketplace”.

Thus, what's open about Linux-based infrastructure expands out to support whole marketplaces.

They Said It

Don't worry about people stealing an idea. If it's original, you will have to ram it down their throats.

One of my many mottos of the past is “Only steal from the best.” When you use someone else's idea that's the ultimate sign of respect. But it's important to say who you're stealing from, because they're the best, right?

Think about it this way: if the water that's piped into your house had DRM on it and only allowed you to use it for showers, how would you wash your clothes? If you were only allowed to make ice cubes, how would you make iced tea? If you had to pay $0.99 every time you wanted a glass of water?

Ideas and hope need to flow like water if a civilization is to continue its ascension toward greatness. Impediments to that flow will stall growth. Fortunately, like a solvent, the culture of open source will continue to expand, will wear away at these impediments, to restore the natural flow of social capital, of ideas, of hope. Those who get this first will rise, and rise quickly.

It's simple. The Internet has won. Why negotiate terms of surrender?