Robin introduces Film GIMP, an open-source Linux tool being used in major motion pictures.
The GIMP probably leads any list of killer applications for Linux. This Photoshop-like graphics package is very popular for retouching still images. However, fewer people are aware of its motion picture cousin called Film GIMP, intended for working on a series of images.
Linux is now being adopted in motion picture production at DreamWorks, Pixar, ILM and many other major studios. Most of this Linux effort involves commercial tools such as the popular 3-D animation package Maya (see the article “Alias|Wavefront Maya 4” in the October 2001 issue of Linux Journal) or obscure internal tools representing an investment of millions of lines of code created by the studios themselves (see “DreamWorks Features Linux and Animation”, August 2001 Linux Journal). Today there is just one significant open-source Linux tool being used in major motion pictures. Let's take a look at Film GIMP and its use at Los Angeles film and television commercial postproduction studio Rhythm & Hues.
R&H programmer Caroline Dahllöf is a lead developer and maintainer of Film GIMP. “We use Film GIMP on all talking animal jobs”, says Dahllöf. “Film GIMP is in use in production by various studios but probably is used most by R&H. Other studios say that they think GIMP is a great idea, but we seem to be the only production house currently developing and supporting it.” Dahllöf would like to see other studios become more involved in development. At R&H, Film GIMP has been used in Harry Potter, Cats & Dogs, Dr. Dolittle 2, Little Nicky, How the Grinch Stole Christmas, The 6th Day, Stuart Little and Planet of the Apes. R&H also creates commercials, such as the familiar Coca-Cola bear commercial. Dahllöf says:
Our big thing is talking animals. We'll create those in 3-D because we like the look much better than 2-D morphing. We'll make a 3-D model and track it to the plate, matching the movement of the live animal with a CG animal head. Then the lighting department projects the frame on to 3-D model. The 2-D department fixes missing background parts as the animal talks.
Stretching has to be fixed with textures. The mouth interior is all CG. Some projects, such as the Coke commercial, are all CG. That's a different technique than making live-action animals talk.
As is typical with production studios, R&H uses not just one tool but a pipeline of tools for 3-D animation and live-action special effects. Before looking at Film GIMP, let's examine some of the proprietary tools in the production pipeline that Film GIMP must seamlessly interact with.
Film GIMP Screenshot—Working on
Film GIMP Screenshot—Visa Commercial
For 3-D modeling, R&H uses an in-house modeling tool called And and Maya. Modeler/TD Yeen-shi Chen explains:
For a TV commercial I am creating a model of a cat wearing a wet suit. The cat model was retrieved from our model library and modified to match the cat in the commercial. The wet suit and diving gear are added later. The entire model was built in And. I create the model in a neutral pose because that makes it easier for the setup people to put a skeleton in it.
Technical Director Jeff McLean explains how R&H uses IC (Interactive Compositing), their internally developed compositor: “A typical task is removing the arms of a skateboarder from a scene in Scooby Doo to be replaced by Scooby in a barrel.” (Scooby Doo is due for release June, 2002.) When McLean replaces a live actor with an animated figure like Scooby, he must not just cover up the actor but handle situations where the actor's movements extend beyond what the animated figure will cover. “I have to do a background replace and repair the missing pieces of the background when I remove an object from a scene”, says McLean.
ICY Screenshot—Canon Commercial
Lighting TD Greg Yepes uses internally created lighting tool Voodoo on projects such as Harry Potter or Canon printer commercials. “When setting up lighting for 3-D objects composited into a scene I'll initially set my virtual lights to an extreme value so I easily can see what I'm doing”, says Yepes. “Later it will go into Wren with more subdued values.” Similar to RenderMan, the Wren software is their internal rendering engine ported to Linux a year ago. “All the apps have gotten ported faster than we had thought”, says Yepes. “I started with DEC Alphas on Cats & Dogs, then the Intel PCs came on-line and really saved us.”
Voodoo Screenshot—Canon Commercial
Film GIMP replaced an internally developed paint tool called Inc. “We like to use Film GIMP to dust-bust”, says Dahllöf. “There might be dust or a piece of hair on the plate either from scanning or from the negative. Normally one will clone/merge from the previous frame or from the same frame to remove the dust.” Film GIMP also is used for rig and wire removal, for example on How the Grinch Stole Christmas. Lighters may use Film GIMP to edit fur control files. “These control files are used by our internal fur program, Fur. Lighters also use GIMP to paint textures and make paint fixes to frames”, says Dahllöf.
“As the studio moves to Linux, more and more people will use Film GIMP on Linux”, says Dahllöf. Because she is evaluating SGI-based commercial paint programs, Matador and Illusion, Dahllöf usually works with Film GIMP running on SGI, not Linux. “I'm looking to see what features I need to add to Film GIMP to make it an equal tool. We need a good paint tool for Linux that our 2-D department can use. We have not found a satisfactory commercial solution.”
“GIMP didn't support any sequence work, which is important for a 2-D artist”, says Dahllöf. “A lot of their work is cloning from one frame to the next in a sequence. So, we added a frame manager.” R&H uses its own proprietary RLL file format. “A major feature of Film GIMP is 16-bit per channel color, and it is compatible with our file format. We do not get any of the color loss when using Film GIMP that we can get with other 16-bit paint packages.”
To get Film GIMP we had to check it out of anonymous CVS. There is no tarball, RPM or deb. You must build it from source. The Film GIMP branch is named HOLLYWOOD:
cvs -z3 -d:pserver:anonymous@gap:/cvs/gnome checkout -r HOLLYWOOD gimp
We had configured our firewall PC named gap to have its port 2401 point to the server anoncvs.gimp.org. If we weren't behind a firewall, we would have specified that server directly in the cvs checkout command rather than our proxy.
After the 18MB download from CVS completed we had to make some minor corrections in order to build. In gimp/plug-ins/Makefile.am we had to delete rll, pts, fm_pts and parsley from the list of directories included in the SUBDIRS variable there. These plugins will not build in Film GIMP and would cause the build to fail.
cd gimp libtoolize --force aclocal automake autoconf ./configure -prefix=/usr/local make ./app/gimp
Technology VP Mark Brown says, “We have 50 Linux machines as desktops now and will have 250 Linux desktops by the end of 2002. We also have a 200-node Linux renderfarm, but that expands or contracts with demand.” R&H wrote their our own virtualized filesystem to support PTS, their production tracking system. R&H uses the ext2 filesystem.
“We decided against actually building our own boxes and have ordered 100 Dual processor 1.5GHz AMD machines with Angstrom Microsystems”, says Brown. “This was the most cost-effective move at this time. All our current desktops are dual PIII. We will support a completely heterogeneous environment. Video cards and CPUs are two of the things we know we can't keep completely consistent at the desktop.”
Film GIMP came about thanks to the patronage of R&H and software development company Silicon Grail. Each hired an OSS GIMP programmer for a year: GEGL designer Calvin Williamson at R&H and GIMP maintainer Manish Singh at Silicon Grail. Silicon Grail founder Ray Feeney explains, “We had done some other open-source projects, such as film recorder drivers, and saw enhancing GIMP as an opportunity to do something with the Open Source community.”
Silicon Grail RAYZ product manager Craig Zerouni says, “We did a little work integrating GIMP into our compositor Chalice as a plugin. But in the end we decided a nonprocedural paint program didn't fit well into a procedural program like Chalice.” Silicon Grail was working with GIMP script-fu to create a series of Film GIMP commands that could be saved in Chalice. However, that work was abandoned when Silicon Grail developers switched to begin development on their new compositor product, RAYZ.
Zerouni feels a true procedural language is needed in the paint program, something like the language in RenderMan. Silicon Grail has lately acquired the Cineon source code from Kodak, including the program Retoucher. “Film GIMP was a useful thing for us to do”, says Zerouni. “We learned a lot about what paint should be.”
GEGL, the GIMP E Graphical Library, is an image-processing library based on GObjects. GEGL developer Calvin Williamson helped develop Film GIMP originally while at R&H, together with Ray Lehtiniemi from Silicon Grail. The next version of GIMP will be 1.4, but Film GIMP continues in development on a branch of 1.0.4. GEGL, whose design supports 16-bit channels, is due to integrate with GIMP 2.0, perhaps two years away. GIMP 2.0 is anticipated to bring the pro features of Film GIMP into mainstream GIMP.
Williamson says his current plan is to write a baby compositor for GEGL to test memory management for large images, multithreading, large composite trees and other heavy-duty professional requirements:
The classes that do image and memory management have been split from the actual image-processing classes. This allows one to write image managers that traverse the graph of ops in custom ways, or write custom caching or memory managers for handling memory management. The classes that hold information about ops as part of graphs, with inputs, outputs, regions of interest (all extrinsic op info), have been separated from image-processing classes (intrinsic op info) as well. This makes graph traversals cleaner for things like multithreading.
“GEGL is still in a very early phase and many classes are under construction”, says Williamson. “There is no official release yet, but you can download it from anonymous CVS. I have made quite a few architecture changes recently.” GEGL is a fully 16-bit image engine for future GIMP and other projects.
Both GEGL and Film GIMP seek volunteers to help with programming. Williamson says PDI, ILM, ICT and Sony studios have expressed interest, but so far have not provided programmers. Williamson welcomes programmers interested in writing image operators, memory management code and working on multithreading to join the GEGL Project. “Filling out the library and writing image operators takes time”, notes Williamson. “GEGL is an important part of the future for GIMP.” For programmers who find the GEGL timeline too far away, Williamson suggests helping with enhancing Film GIMP for the next year or so.
Film GIMP maintainer Dahllöf says, “We want to enhance Film GIMP by adding more tools for artists working on a sequence of frames. We want to make it easier for them to paint and clone from one frame to another one.” R&H uses a proprietary flipbook player called Flicks. A feature missing from Film GIMP is a flipbook playback mode so users can detect flickering. “Artists do use filters, but some of the filters in the main branch of Film GIMP are not useful in motion pictures. Artists want more control over filters.”
On the topic of open-source software, R&H principal software engineer Green says:
It's a big debate about releasing more of our proprietary software as open source, beyond Film GIMP. Most of our software would be difficult to use outside this building. It is tied to our production tracking system, PTS. Nothing will run without that.
Green says the biggest pro argument is to save on the high cost of training. New hires at R&H spend their first month just doing training. Having a pool of talent emerging from the universities already trained in R&H tools would help.