Guard Against Data Loss with Mondo Rescue

Hugo Rabson

Issue #92, December 2001

Hugo shows how to use Mondo to restore your system in the event of catastrophic data loss.

Every computer user has lost data at some point. Unfortunately, the experience rarely teaches the user to backup his or her data regularly. Mondo Rescue creates one or more bootable Rescue CDs based on your filesystem. In the event of catastrophic data loss, you will be able to restore your entire system. Mondo supports Lin/Win dual-boot systems, RAID, ReiserFS, ext2, ext3, XFS, JFS and VFAT.

I wrote Mondo because no other program did what I wanted, nor was any program as easy to use. I back up, wipe, format and restore my own system at least three times a week using Mondo's latest release. Mondo is the only backup regime that I use.

Background

Mondo is a thriving open-source project; it can restore your OS and data from bare metal, and data loss has not been reported since May 2000. The project is coming along nicely.

In the beginning, Mondo was created to back up Windows-only systems and provide disaster recovery facilities for Windows users. While running a small computer store near Nashville, Tennessee, I realized that many of the people that bought computers from us were bringing them back to have the disks reformatted and the applications re-installed. Was this because they did not know how to maintain their own PCs? Was our hardware to blame? Either way, we needed a way to restore a fresh copy of Windows onto a typical PC, unattended, in 10-15 minutes. We were using Norton Ghost at the time, but Ghost didn't let us restore selectively; it was all or nothing. Also, Ghost tended to crash when copying Linux partitions.

Our idea was to run Linux on a CD but have Windows archived in big tarballs on said CD. Well, to cut a long and tortuous story short, it worked. I even managed to write code that would create a Windows-bootable VFAT partition.

In March 2001, an employee of Hewlett-Packard (Grenoble) became interested in the project. Bruno Cornec, a skilled programmer in the server division, began to look at the code. He started to play with it and improved it in numerous ways. Between September 2000 (when the project was shut down) and March 2001, two new versions of Mondo were released. Both were made possible almost entirely by Bruno Cornec, Maciej Kulasa and other contributors.

In June 2001, I built a new computer and got to work. By July 4, I had fixed all the major bugs, and Mondo v1.00 was released (no jokes about Independence Day, please).

How It Works

Mondo is just a bit shell script that wraps around two other tools: Mindi and afio.

Mindi, aka Mindi-Linux, is a mini-distribution that generates boot disks from your kernel, modules, tools and libraries. It is more likely than a generic boot disk to be binary-compatible with your existing software because its tools are copied straight from your hard disk. It also can generate an El Torito 2.88MB boot disk image. Mondo uses Mindi to create a 2.88MB boot “disk” and associated data “disks”, which are incorporated in each Mondo CD. Whichever modules were loaded at backup time are reloaded at boot time. So, in theory you will boot into almost the same environment you were in when you backed up.

Mondo-Archive works something like this:

  1. It squeezes your files into several tarballs, each 5-10MB in size.

  2. It puts those tarballs in a directory.

  3. It puts an El Torito boot floppy image and associated data files in that directory.

  4. It points mkisofs at that directory and pipes its output to cdrecord, which causes a CD to be burned that contains a copy of that directory and its contents.

  5. It repeats the above steps for N CDs, N being the number of CDs it takes to accommodate all your files.

Kernel Configuration

Mindi v0.38 comes with a 2.4.7 fail-safe kernel, in case your kernel does not support all the features required by a boot disk. This is not Mondo being picky; this is Linux being awkward. Some kernels are just not suitable for boot disks. Novice users should stick with their default kernel and make Mindi use its own kernel by saying no when asked if they want to make a boot disk with their own kernel.

Advanced users should make sure that their kernel includes the following, built-in: CD-ROM support, IDE CD support, IDE support, initrd and RAM disk support, floppy disk drive support, stable loopfs support (which means it really needs to be 2.2.17 or later, or 2.4.5-ac10 or later) and ISO9660 support.

In case your kernel does not support these features, you should probably use Mindi's own kernel until you are confident enough to build your own kernel. If you find that Mindi's kernel does not support something you need (e.g., XFS) then please let me know. I will probably add the feature to the next release of Mindi.

Getting and Installing Mondo

To install the program, go to www.microwerks.net/~hugo and download Mondo and Mindi. The latter is part of the former but was forked because Mindi also creates standalone boot disks based on your kernel, modules, tools and libraries. Installation instructions for both tools are provided on the Download web page.

RPM users have it easy; they simply need to download Mindi's RPM to /tmp, download Mondo's RPM to /tmp and then type the following:

rpm -Uvh /tmp/mondo-1.13-1.i386.rpm /tmp/mindi-0.39-1.i386.rpm

Tarball users have a slightly harder time; they must download Mindi's tarball to /tmp, download Mondo's tarball to /tmp and then type the following:

cd /tmp
tar -zxvf mindi-0.39.tgz
cd mindi-0.39
./install.sh
cd ..
tar -zxvf mondo-1.13.tgz
cd mondo-1.12
./install.sh

Other Tools Needed by Mondo

Some distributions lack certain crucial packages. The packages most often missing are afio, cdrecord, bzip2, libnewt0.50, libslang1 and mkisofs. Some users may have to create a gawk-to-awk shortcut. You can find these tools on the web site of the vendor of your distribution.

Make a Test CD

Making a test CD is a good idea because the new user can try it and not mess up his or her system. First, be sure that Linux knows how to use your CD writer. Then, run mondo-archive.

To find your CD writer, type

dmesg | grep CD

If your CD writer is an IDE device, it will show up here as /dev/hdX, X being a letter between a and h. If SCSI emulation is properly configured, you will see your CD writer listed when you type

cdrecord -scanbus
If your CD writer is properly installed, you will see
0,0,0 --- JoeCamel 4x CD Writer
or something similar. The 0,0,0 number to the left of the device description is the SCSI device where the writer can be found. Write this number down.

If you want your rescue CD to include certain special programs, e.g., your copy of BRU, add the file and its config files to /usr/share/mindi/deplist.txt by hand. Mindi will find the libraries and add them for you.

Run Mindi to create some boot disks just to make sure Mindi works properly on your system. Type

cd /usr/share/mindi
./mindi

If your kernel is too large (more than about 900KB), you cannot make boot floppies, although you still can make a bootable CD image. The easiest way to test Mindi in either case is to press N to “Create boot floppies?” and Y to “Create iso image?” Then use cdrecord to make a bootable CD-R or CD-RW. Type

cd /root/images/mindi
Then choose one of the following calls to write the CD, depending on whether the disk in the drive is a CD-R or a CD-RW. Please replace x,x,x with your writer's SCSI node. For CD-RW, type
cdrecord blank=fast dev=x,x,x speed=2 mindi.iso
and for CD-R, type
cdrecord dev=x,x,x speed=2 mindi.iso

Close all applications; reboot from the CD instead of the hard disk. (You may have to edit your BIOS settings to make your computer try to boot from the CD before the hard drive.) If your computer boots okay from the CD then you know Mondo also will generate a bootable rescue CD reliably. Of course, an ideal rescue CD will use your own kernel. I recommend that you use your own kernel if possible, to minimize the risk that the boot CD won't support your hardware or filesystems, etc.

Finally, to do a complete backup, type

cd /home
mondo-archive --burn-cds 2 0,0,0 --comp-level 9

The 2 indicates that you are writing at 2x speed. If you are burning to CD-RWs, type

mondo-archive --burn-cds 2 0,0,0 cdrw --comp-level 9
After running the command, insert a blank CD-R(W) into the drive and leave the PC running. That's all.

I always choose the maximum compression level (9) because I start Mondo and then go to work. When I come home, I insert the second CD-RW and wait a half an hour. That is a day's backup.

The default compression level is 3. If you are in a hurry, use --comp-level 1 to speed up the backup process. You will use more CDs that way, but it should take less time to run.

If Mondo does not find a CD in the drive when it tries to write files to the CD, it will pause with a Retry/Fail/Abort message. If you insert a CD and choose Retry, it will retry as if nothing had gone wrong. If you choose Abort, the program will stop. If you choose Fail, the program will skip that CD but continue the backup process. Mostly, you should choose Retry.

If there are specific paths that you do not wish to backup, you may exclude them with

--exclude-paths /foo /bar /xanadu

If you want to include only certain paths, use --bkpath /home. So, if you want to backup only your home and boot directories but exclude the communal MP3 folder, use this:

mondo-archive --burn-cds 2 0,0,0 cdrw --exclude-paths /home/MP3s /home/WAVs /home/secret
If you do not want to burn the CDs immediately but would rather create ISO disk images to be burned later, do this:
mondo-archive --isodir /root --bkpath /home /boot --exclude-paths /home/MP3s /home/WAVs /home/secret
This will create 1.ISO, 2.ISO, etc., and save the files to the /root directory.

Before running Mondo-Archive, be sure to add some files to /usr/share /mindi/deplist.txt, run mount to make sure you have mounted the partitions that you want be backed up and run df to determine backup size/compression/CDs needed.

You can speed up the compare process by switching to another terminal after booting and running ide-opt. Use Alt-left cursor key and type ide-opt. This enables DMA and other good stuff.

To compare the backup against your live filesystem, please boot from the CD and choose compare mode (type compare and then press Enter). Check /tmp/mondo-restore.log after the compare cycle to see which files do not match. Aside from the initial teething troubles you might encounter with making boot disks from your kernel (some kernels are not appropriate for boot disks and have to be recompiled), you are likely to discover that Mondo is quite boring. It does what it says it does. It squeezes all your files onto your CD-R(W)s, and it restores them again if necessary. It partitions your drives, formats them, restores the data and runs LILO to set up your boot sector.

Bare-Bones Restore

Imagine that your hard drives happen to be wiped, deliberately or accidentally. Or, imagine that you want to clone your existing operating system. In either case, you want to run in Nuke Mode. After booting from your Mondo CD, type nuke, press Enter and watch the show. That's it.

If you want to see exactly what Mondo is doing while it is restoring, press Alt-A and type

tail -f /tmp/mondo-restore.log

to monitor its progress in detail.

Selective Restore

If you want to restore only some files, or if you do not want to prep/format your drives, then you should boot into interactive mode. Type interactive and then press Enter at boot time. You will be asked to say yes/no to a range of questions: do you want to partition your devices? Do you want to format them? Do you want to restore everything? Do you want to restore something? Do you want to run LILO to set up your boot sectors?

Interactive mode is for people who have lost a subset of data from their live filesystem or perhaps who have lost some data from their latest backup and want to restore a subset of data from an earlier backup.

Using Expert Mode

When you have backed up your system, booted from the CD in compare mode and verified the archives, you are in a position to experiment with your system. You could move partitions around, resize them, enable or disable RAID, play with other boot loaders, etc. That is what Mondo is really good for. The greatest threat to a typical Linux installation, in my opinion, is a careless root user.

If you want to try some of these tricks, boot from your Mondo CD and choose expert mode. This will drop you to a shell prompt. Edit the mountlist file. Then, type mondo-restore and choose Interactively from the pop-up menu. The mountlist is a text file, /tmp/mountlist.txt, located on the RAM disk after you boot from the Mondo CD. It lists the various partitions that will be created, their sizes, their mountpoints and their formats. To change the size or layout of your partitions, just edit that file with pico /tmp/mountlist.txt (or use your favorite editor). Save and close by pressing Ctrl-X and then Enter.

Here is a sample mountlist. The sizes are in kilobytes, so count the zeros carefully when modifying the values. Remember, the new layout will not take effect until you run mondo-restore to repartition and reformat the drives:

/dev/hda1    /mnt/windows       vfat        4096000
/dev/hda2    swap               swap        256
/dev/hda3    /                  ext2        8192000

To change the root partition's size and format, simply alter the relevant fields:

/dev/hda1    /mnt/windows       vfat        4096000
/dev/hda2    swap               swap        256
/dev/hda3    /                  reiserfs    16384000
Or, you could move from using multiple primaries to using a primary partition (hda1), an extended partition (hda2, created/handled by Mondo) and several logical partitions. Note the new /dev/hdaN entries:
/dev/hda1    /mnt/windows       vfat        4096000
/dev/hda5    swap               swap        256
/dev/hda6    /                  reiserfs    8123000
/dev/hda7    /usr               reiserfs    4099000
/dev/hda8    /home              reiserfs    4099000
If you have added a second hard drive (e.g., primary slave) then you could move some of your partitions to that drive. See below and note the changes:
/dev/hda1    /mnt/windows       vfat        4096000
/dev/hda2    swap               swap        256
/dev/hda3    /                  reiserfs    81422000
/dev/hdb1    /home              reiserfs    9481000
/dev/hdb5    /usr               reiserfs    16384000
/dev/hdb6    /tmp               reiserfs    1589000
It is slightly more complicated to move to RAID because you have to create an /etc/raidtab file. You can do this from within Expert Mode. Just type pico /etc/raidtab and create a good raidtab file. (That is beyond the scope of this article.) Then, replace the conventional device with a RAID device (/dev/mdN):
/dev/hda1    /mnt/windows       vfat        4096000
/dev/hda5    swap               swap        256
/dev/md0     /                  reiserfs    16384000
After editing the mountlist, run mondo-restore. When asked if you want to partition and format the drives, say yes. You may want to restore the data or run LILO to initialize the boot sector, depending on what you are doing. If you are simply testing a new partition layout, you probably want to say no when answering the other questions. Otherwise, say yes.

Conclusion

Good backup software must be well documented, well tested and easy to use. If the backup software is too fiddly then the user simply will not bother. If it is unreliable, then the user will be lulled into a false sense of security, only to have his or her hopes dashed when the crunch comes. I back up my own system with Mondo two or three times a week. I restore regularly as part of the test regime.

If you want to participate, please download the program and join the mailing list. I would love to hear from you.

Other Uses of Mondo

email: hugo@firstlinux.net

Hugo Rabson is a 26-year-old Briton living and working in Nashville, Tennessee. His primary qualification is in business, but he has been writing code since puberty. He currently works for WebMD. Mondo is his attempt to give something back to the Linux community, and he gladly will accept all comments, suggestions and patches.