Linux as a Video Desktop

For the Emmy award-winning movie Titanic the production studio Digital Domain used Linux on a network of more than a hundred DEC Alpha machines to render the special effects. However, they used 350 SGI machines running IRIX and a hundred DEC Alphas running Windows NT for the artistic aspects of the production. Linux was judged not-ready in 1997 for the video desktop. What's developed with Linux as a desktop operating system for video production in the last few years?

For Linux to be a player in the realm of television and motion picture production it must have stunning graphics capabilities, high-performance disk I/O with support for the very large file sizes video needs, integration with analog and digital video input/output devices, and the actual video applications themselves. As these capabilities are becoming available on Linux, the companies that develop video applications (including the company I work for) are looking at Linux in a new way.

Because Digital Convergence seems like too much of a mouthful for a title, and Linux Multimedia too lame, we'll call the column Media/GFX. GFX is an abbreviation of “graphics” used in television news rundowns and scripts, a term I picked up years ago as technical director for news production at an NBC television station. Media/GFX will tell about our travails with Linux over the next year as we install multimedia applications and make progress on the programming front and will discuss what it is like to develop multimedia applications for Linux from scratch. My own area of expertise is video software design using C++ and Java, and that's always interesting (and frustrating) on any platform. Video applications are always some of the most challenging because of the demands they make on the CPU, disk and I/O systems.

Companies that develop software for their livelihood need to be practical in the operating system they choose. It needs to be popular, capable and not too hard to develop software for. Every few years that platform tends to change. For example, during the past four years my software development has been mainly on Windows NT, with some code ported to Solaris and Alpha for servers. A major motivation to use Windows is the vast array of relatively inexpensive hardware available for it, including MPEG-1 and MPEG-2 video compression cards. Before my switch to WinNT it was Sparc20 Solaris with a Parallax Motion-JPEG card, and before that a simple SGI Indigo running IRIX. If you are a video software developer it is best not to get too attached to an operating system.

The Macintosh was perceived as the graphics platform of choice for many years but has been eclipsed by the sheer popularity of Windows. Windows 98 and Millennium Edition versions are the dominant platforms for games utilizing high performance graphics. However, stability and security concerns with DOS-based Win9x tend to preclude its use in high-reliability broadcast scenarios. For the same reason, Win9x is broadly prohibited for Department of Defense users. The DoD is a major consumer of graphics systems, usually WinNT or Solaris. Windows NT/2000 is also a major player in video workstations. An OS design influenced by VMS (which was also created by the same architect), WinNT/2k is more like UNIX than the DOS-based Windows versions, even to POSIX compliance. In the high-end, SGI with its IRIX flavor of UNIX has long dominated motion picture production video workstations. However, WinNT/2K has made inroads.

Earlier this year SGI announced they are moving to Linux in a major way. The big news is graphics, the area SGI has built its reputation upon. In collaboration with video chip manufacturer NVIDIA, SGI Intel-based Linux workstations models 230, 330 and 550 can, according to SGI, draw over 17 million triangles per second and up to 540 megapixels per second. Their VPro graphics subsystems are 32MB or 64MB DDR AGP 4X boards. Perhaps like me, you're not ready to rush out and buy an SGI workstation. You don't have to. The graphics improvements have been made open source and rolled into XFree86. The graphics capabilities of Linux have taken a quantum leap forward!

Another area of video capability where SGI is making a significant contribution is file systems. A DV IEEE-1394 Firewire video transport stream moves at about 30MBs (typically advertised as just 25MBs, but that's counting video only, not audio). At that bit rate you will bump up against the 2GB file size limit in Ext2 in less than nine minutes. To contain one 22-minute television program you need to create a 5GB file. For a 100-minute movie, make that 23GB.

XFS is a high-performance journaled file system capable of handling files as large as 9,000 petabytes that would hold a DV file lasting 837 centuries. XFS has been ported from SGI IRIX to Linux and was released in September in beta. Journaled file systems treat disk activity as database transactions and are able to roll back to a stable configuration if interrupted by a crash or power-down. No fsck. XFS isn't the only high-performance Linux file system available. Other journaled file systems (not from SGI) include JFS, ReiserFS and Ext3FS. Journaled file systems don't have to handle large files. In fact, Ext3FS, built upon Ext2, isn't suitable for large video files because it inherited the same 2GB limit.

SGI GLX is the glue code that connects the board-level OpenGL graphics language with the X Windows System. The GLX Client Library presents the popular OpenGL API to applications, performs indirect rendering via X11 network protocol, and high-performance direct rendering where the client display and server are the same machine. The Precision Insight implementation is called the Direct Rendering Infrastructure (DRI) and uses GLX, Mesa 3.1, XFree86 4.0 and Linux kernel modifications. The work of porting GLX to Linux was carried out by Precision Insight, sponsored by SGI and Red Hat.

OpenGL is a platform-independent scene description language, implemented in the hardware of most high-performance graphics cards and even in cheap PC cards costing less than $100 today. It is the most widely available 3-D API (Linux, Mac, Win32 and others), beating Microsoft Direct3D and 3dfx Glide. OpenGL enables 3-D application programmers to get their software to work on any hardware without writing their own drivers for every 3-D board out there. Being a standard 3-D API of course means that OpenGL can be used for 2-D as well.

A game designer uses OpenGL to describe what to draw as a set of graphics primitives (shapes in space) and texture maps (images). Having hardware handle these 3-D calculations can vastly speed up screen rates in virtual reality interactive games such as Quake (http://www.quake3arena.com/).

Figure 1. Screen Shot from Quake III

SGI's reference implementation of the OpenGL API is widely used to develop hardware drivers for their systems. Mesa, a 3-D graphics library with a work-a-like API similar to that of OpenGL, is the glaring exception. Mesa (http://www.mesa3d.org/) was developed independently to circumvent the OpenGL commercial license. SGI has since made OpenGL, GLX and XFS open source. SGI Open Inventor is a popular object-oriented 3-D toolkit based on a 3-D scene database, also now open source. Open Inventor is built on top of OpenGL and supports PostScript printing. See the SGI open-source web site at www.sgi.com/developers/oss/.

With high-performance graphics only now becoming available, graphics applications for the Linux platform are still catching up with their SGI IRIX and Windows NT counterparts. The GIMP, of course, is highly regarded as a replacement for Photoshop. But, did you know you can use GIMP to overlay graphics in Broadcast2000 [see Adam Williams' “Moviemaking on a Linux Box? No Way!” in the January 2000 issue—Editor], a Linux video nonlinear editor? Broadcast2000 manipulates uncompressed 720x480 video captured using a Video4Linux-compatible video card such as the inexpensive Haupaugge WinTV or cards from Linux Media Labs. Audio is 48K stereo sound from an OSS-compatible audio card. Starting with kernel version 2.2, Linux supports the DV IEEE-1394 Firewire protocol.

Many graphics programs are available for the Linux platform in addition to the GIMP. Blender is a popular animation package to create stunning three-dimensional graphics. Houdini was the first major commercial 3-D animation package available on Linux. Ported from SGI IRIX, Side Effects Software's Houdini has been used extensively for effects animation at Digital Domain and other major studios. It was used for effects in Titanic and in X-Men. At $17K US, Houdini is not free, but a 30-day evaluation version is available.

During the coming months we will investigate every Linux multimedia-related item we can:

An expert with Linux might be able to cover these easily but not with the same point of view. With almost no experience with Linux personally (although quite a bit with Solaris and Windows) we can expect to make some of the dumb goofs that any Linux newbie would make. By going through this process from the start, we can learn together.

We are right at the start of our Linux multimedia installation. A few days ago a new Maxtor 20GB 7200RPM hard drive was still in the box next to my desk waiting for a fresh install of Win98, Win2k and Debian Linux on it. We're creating a triple-boot system because we still have our regular Windows development work to do for our existing products. Also, it is our perhaps naïve plan to develop the same applications across Windows and Linux from a single-code base. That will be some trick given the differences between the X and Win32 GUI APIs.

Our PC contains a WinTV card for watching/capturing television and a PyroDV IEEE-1394 Firewire card to interface with a digital camcorder. When we get all that working right we will add an ATI All-in-Wonder and maybe a Compaq iPAQ handheld that we want to do some multimedia development with. (We too saw the PocketLinux iPAQ running at the San Diego USENIX convention, very cool. See http://www.handhelds.org/.)

Linux continues to excel as a network server. The BBC uses Linux to serve web pages, RealMedia and their digital text service. Victoria's Secret relied on VA Linux servers to stream their Cannes 2000 Fashion Show, one of the largest live webcasts in Internet history, more than two million viewers worldwide. With support from SGI, NVIDIA, Red Hat, Side Effects Software and many others, Linux is expanding beyond network servers and animation render farms into desktop video production. Next month we'll boot our new system and start finding out just how good the Linux is becoming as a video desktop.