In the first part of this three-part series on real-time audio synthesis, we take you through the history and basis of RTcmix.
RTcmix is a software package that performs audio synthesis and effects processing. Basically, it's a real-time version of the Cmix language. Standard control over program execution is achieved via a C-style scripting language called MINC (MINC Is Not C). It has been open source long before the phrase was coined. That fact has been key to its evolution.
There have been literally dozens of RTcmix hackers over the past 20 years. I've done my best to give some credit as it relates to this article. You can also check the AUTHORS file included in the recent distribution (see “Getting and Installing” below).
Cmix was derived from the MIX program, a 20-track mixer written by Paul Lansky (at Princeton University) in 1978, which ran on VMS in FORTRAN on IBM mainframes. Synthesis abilities were added, and Paul ported Cmix to run on a PDP11/34 in 1983-84 under BSD2.9 UNIX. In 1985, he moved it over to Ultrix on a DEC MicroVAX. He and Lars Graf added the MINC parser in 1987, with help from Brad Garton and Dave Madole on various parts. Cmix lived happily under the NeXt (and other UNIX systems) for several years with its user base primarily in academic institutions. It has been and continues to be used both for instruction, computer music composition and research development.
Doug Scott and Paul Lansky ported an initial real-time version to SGI in 1995. While sometime in 1993, as a student and research assistant at Columbia University's Computer Music Center (CMC), I took a first stab at a Linux port which had to use Sox to unswap big-endian files. Boy was that fun. In late 1995, Brad Garton (CMC director) and I created a real-time version with a scheduler. RTcmix was born, originally running under IRIX. Porting to Linux basically entailed swapping out the IRIX audio API with OSS (Open Sound System). The initial IRIX version was presented at the International Computer Music Conference (ICMC) '97. Shortly thereafter, Luke DuBois (also at the CMC) added an API to make writing TCP socket data easier. Doug Scott set up a mechanism to dynamically load instruments at run-time in 1997. John Gibson (at UVA) cleaned up the business of reading and writing different audio file formats in 1999 with an interface to Bill Schottstaedt's (at Stanford University) sndlib. John and I recently added two new abilities: routing audio via an internal “bus” mechanism and multichannel audio support. This work was presented this past year at the Society for Electoacoustic Music in the U.S. (SEAMUS) Y2K conference in Denton, Texas. Work also continues on controlling synthesis parameters in real time, creating various interfaces, and more robust socket support.
Setting up CVS (Concurrent Versions System) to help manage code has facilitated recent collaborations and development. The main core of RTcmix is released under the GPL (GNU Public License), with some different licenses for various components (e.g., instruments and effects) depending on their origin.
In order to make use of RTcmix's real-time audio abilities, you need to install and set up the OSS audio drivers for Linux (see Resources). Support for ALSA (Advanced Linux Sound Architecture) remains untested but is planned. If you want to use RTcmix as a real-time audio processor, you need to make sure your audio card supports full-duplex operation. The Linux version of RTcmix can be downloaded via ftp from:
presto.music.virginia.edu/pub/RTcmix
The latest version as of this article is 3.0. The complete package exists in separate packages. RTcmix-3.0.0.tar.gz: the core engine and scheduler. insts.jg-3.0.0.tar.gz: instruments written by John Gibson. insts.mch-3.0.0.tar.gz: multichannel instruments. insts.std-3.0.0.tar.gz: standard suite of instruments. [At publication time, current versions of all of these packages is 3.0.2—ED]
I should probably comment on some terminology here. An RTcmix “instrument” can be either an effects processor (e.g., DELAY) or a synthesizer (e.g., FM synthesis).
Instrument packages should be unpacked into the same directory as the main code. So if you unpack the main code (RTcmix) to /usr/local/src, you'll end up with /usr/local/src/RTcmix-3.0.0. Untar instrument packages into that directory. In the case of insts.std.tgz, you'd end up with /usr/local/src/RTcmix-3.0.0/insts.std-3.0.0. You can unpack the main code into any directory you like (e.g., your home directory or anyplace else).
In order to keep instruments and RTcmix versions consistent, we've added version tags to their directories. To make compilation simpler, you should symlink these to default names. For example, insts.std.tgz will untar into insts.std-3.0.0. You should symlink this as:
ln -s insts.std-3.0.0 insts.std
Once you've gotten everything unpacked and linked, you can set things up for compilation. There's no ./configure utility here. You have to edit makefile.conf in the RTcmix root directory. Only a few lines need changing. Reading from the top of the file:
# Change stuff in here as needed # The directory containing the RTcmix directory TOPDIR = /usr/local/src # The name of the RTcmix directory CMIXDIR = $(TOPDIR)/RTcmix-3.0.0 # The dir that will contain links to the # instrument dynamic shared objects LIBDESTDIR = $(CMIXDIR)/shlib # add new instruments to this list (e.g., # insts.std, insts.jg) INST_DIRS = insts.base #insts.std insts.jg insts.mchThe only things you really need to potentially change are TOPDIR, depending on where you've unpacked RTcmix, and INST_DIRS, depending on which instrument packages you've downloaded.
Once you've made the appropriate changes, you can do a make && make install from the top-level RTcmix directory. This will compile everything and put a CMIX executable in TOPDIR/bin and shared libraries in TOPDIR/shlib. Make sure the bin directory is part of your PATH.
The MINC scripting language provides both a powerful and easy-to-learn interface to RTcmix. More terminology: a MINC script is referred to as a scorefile. If you understand C, using MINC will be trivial. Some major differences are no semicolons at the end of each line and no functions (sorry). The art of using MINC (RTcmix) to make music is the primary focus of several semester-long college and graduate courses offered at several higher education institutions around the world. There's no way I can do it complete justice here. Instead, we'll take a look at some scorefiles which illustrate various features.
To execute a scorefile, simply pipe it into CMIX with stdin.
CMIX < scorefile
from the command line will fire it off. Internally, this syntax passes the scorefile to a parser (compiled by yacc/bison) which in turn passes events to a scheduler. When used in real-time performance mode (i.e., with external control sources), the RTcmix parser listens to a TCP socket, parsing data on the fly. In this instance, parser and scheduler are implemented using the linuxthreads library. The following scorefiles are included with version 3.0 of RTcmix and were designed by John Gibson.
Listing 1 illustrates a reasonably simple scorefile which uses two RTcmix instruments. There are dozens more. WAVETABLE, as the name implies, is a wavetable lookup instrument. REVERBIT, also descriptive, runs a reverberation effect on an audio stream. In this example, the output of WAVETABLE is piped into REVERBIT, then out to your audio device and/or an audio file.
Let's step through the scorefile a bit:
rtsetparams(44100, 2)
This command is necessary for all RTcmix scorefiles. It sets up the audio device to run in stereo at 44100hz. An optional third parameter allows you to configure the audio buffer size to reduce latency in performance (e.g., when controlling RTcmix from another application). Default is 8,192 frames.
Next we declare which “instruments” we're going to use:
load("WAVETABLE")
load("REVERBIT")
As mentioned above, RTcmix supports the dynamic loading of libraries. These commands load the instruments at run-time. This allows for a smaller memory footprint than loading all the instruments simultaneously. It also allows you to write a score that uses multiple instruments.
The next few lines are commented out. Uncommenting them will allow audio to be written to a soundfile as well as the audio device. The set_option command (de)activates various RTcmix features. For example: set_option("AUDIO_OFF") will turn off writing to the audio device—useful when your scorefile forces your computer to stretch beyond its means.
In order to map the audio from one source to another, eventually to your speakers or a soundfile, we call:
bus_config("WAVETABLE", "aux 0-1 out")
bus_config("REVERBIT", "aux 0-1 in", "out 0-1")
As mentioned earlier, one of RTcmix's newest features is the ability to route audio streams. In an attempt to keep tradition with established mixing convention, we've adopted a “bus” paradigm. Internal buses are labeled as “aux”, which can be read from (as “in”) or written to (as “out”). Input devices are labeled “in”, and output buses (audio devices) “out.” The convention for bus designation is: bus_type “number” [in or out for aux]. So “out 0” would be an output channel (e.g., a speaker) and “in 0” a line in or mic. This is not to be confused with “aux 0 in” which is the specification of an internal bus that is being read from by a particular instrument.
So, the statements above first route WAVETABLE's output to aux (internal bus) 0 and 1. Then REVERBIT's bus_config is set up to read from aux 0 and 1 (WAVETABLE's output) and write its output to a stereo audio device (out 0-1). It is possible to have multiple bus configurations for the same instrument. Each inherits the most recent call.
The next line in the score sets a global variable totdur. Variables are auto declared at init time (i.e., when the RTcmix parser reads your file). There are no reserved global variable names.
In order to set “control rate” update of internal synthesis parameters (e.g., amplitude envelope updates) we use the reset command. So reset(2000) updates 2,000 times every 44,100 samples—every 20.5 samples.
The following two lines are a bit less self-descriptive:
setline(0,0, 1,1, 5,0) makegen(2, 10, 10000, 1,.5,.3,.1)
As with many packages that have developed over a number of years, RTcmix has its share of legacy issues. Unit generators in Cmix are defined by the makegen statement. Makegens are used for various internal aspects of a particular instrument: an amplitude envelope, waveform for a lookup, pitch vibrato, and so forth. The setline command is a related statement, used specifically to set an amplitude envelope. The time-value pairs set an amplitude of 0 at time 0, 1 at time 1 and zero at time 5. Time in this sense is relative and will be stretched accordingly to the duration of your “note” (see below). The WAVETABLE instrument needs some kind of waveform to read in order to make sound. It stores this information in “slot 2”. Slot 1 is used for the amplitude envelope defined by the setline. Yes, you could substitute a makegen command for that setline (gen24). Gen10 specifies loading a sinusoid into the respective slot (in this case 2). The size (in samples) of the wavetable in this case is 10,000. Remaining arguments are amplitudes for successive harmonics. That's about as complicated as it gets, at least in this article.
After assigning some more global variables, we seed the random number generator with:
srand(3284)
The actual music making is done by the following lines:
for (st = 0; st < totdur; st = st + .45)
WAVETABLE(st, dur, amp, freq, random())
This is a simple for loop which executes until st is greater than
totdur (defined earlier). The parameters to WAVETABLE (as well as
all other RTcmix functions and instruments) are referred to as
p-fields. In the case of WAVETABLE:
p0 = start time
p1 = duration
p2 = digital amplitude (in this case between 0 and 32767)
p3 = frequency (in hz)
p4 = stereo spread
The last few lines of the score reset the amplitude envelope with setline(0,1, 1,1), because we want a slightly different one for our REVERBIT function, which we call with: REVERBIT(st=0, insk=0, totdur, amp, revtime, revpct, rtchandel, cf).
The p-fields for REVERBIT are:
p0 = start time
p1 = inskip, which is the time to start reading input. (Use 0 for bus_configs; some other value when reading in from a file.)
p2 = duration
p3 = amplitude multiplier (generally between 0 and 1)
p4 = reverb time
p5 = reverb percentage
p6 = right channel delay
p7 = cutoff frequency for low-pass filter (in hz)
p8 (optional) = apply DC blocking filter (boolean)
That's it. Simply type the following to execute the score:
CMIX < WAVETABLE_REVERBIT_1.scoListing 2 is a musically simpler example, but illustrates the powerful ability of RTcmix to process an incoming audio stream in real time.
To set up our audio device for simultaneous reading and writing, we use the set_option command again:
set_option("full_duplex_on")
It's necessary to have this command executed before the rtsetparams call, as it sets an internal flag that rtsetparams uses when performing actual setup of the audio device. As before, we set up the device for 44K stereo, but this time decreasing the internal buffer size for real-time performance to 512 frames.
To tell RTcmix to read from the audio device, we use:
rtinput("AUDIO", "MIC")
The MIC argument is necessary only in IRIX versions of RTcmix. We could just as well say:
rtinput("some_file.aiff")
to read an audio file instead.
RTcmix is an incredibly powerful and flexible package because of the work of many people, and possible because of its open-source nature. Different features have been added by different people over the years. It continues to evolve. The Linux operating system has been an excellent platform both for development and performance. With the recent advent of high-end, multichannel digital audio card support for Linux (e.g., the RME Digi96 series), the abilities of RTcmix grow in conjunction with the OS.
This article only scratches the surface of RTcmix's potential. Future articles will discuss in greater depth the process of writing RTcmix instruments and controlling them in real time (e.g., with nice open-source GUI packages like GTK).
In the end, it remains amazing to think that a computer, operating system and software package can work together well enough to make music. It is, however, by no means new thinking. Ada Lovelace apparently debated at length with Charles Babbage the virtues of this “new computing device”. His contentions had to do with an unbeatable chess player, hers about a device that could compose and create music of any type or degree of complexity.