Introducing AOLserver

Apache, the well-known HTTP server, is something of a poster child for open-source software: it is popular, stable, flexible, secure, portable, extensible and compliant with internet standards. I've been using Apache since it was first released, and it's a joy to work with.

But given that there are multiple open-source operating systems, editors, databases and programming languages, it shouldn't come as a surprise to hear that Apache isn't the only open-source HTTP server. What is surprising is that one of the alternatives comes from America Online, the same company that sponsors Mozilla, the open-source web browser.

AOLserver offers many of the same features as Apache: it is released under an open-source license, is easy and flexible to configure and offers an API for writing plugin modules. But AOLserver has a fundamentally different architecture from Apache, which often makes it a more efficient choice. Moreover, AOLserver includes a built-in Tcl interpreter, multiple threads, database API and database connection pooling. If your web site uses a lot of database connections, then it's worth looking into AOLserver as an alternative to Apache.

This month, we look at AOLserver as a lead-in to several articles about the open-source OpenACS (Open Architecture Community System) web application framework. While AOLserver is not an absolute requirement for OpenACS, it is the standard and expected way to configure and install the system.

AOLserver began as NaviServer, written and sold by a pioneering company called NaviSoft that offered high-quality, client- and server-side tools for web publishers. AOL has often bought companies that have developed interesting technology; in the case of NaviSoft, AOL bought them for their server- rather than for their client-side tools.

AOLserver probably would have remained relatively unknown were it not for a combination of events: AOL made binary copies of the program available at no cost via the Internet, and Philip Greenspun began work on the ArsDigita Community System. AOLserver makes it easy to create high-performance, database-backed web sites; the fact that ACS made heavy use of relational databases meant that AOLserver was a perfect fit.

But while AOLserver was free of charge, the source code still was unavailable to the general public. In 1999, Greenspun helped negotiate a deal that resulted in the release of AOLserver 3.0 under the AOLserver Public License, which is essentially identical to the Mozilla Public License.

ArsDigita itself effectively went out of business earlier this year, with its remaining staff and assets becoming part of Red Hat Software. But the original ACS software lives on in the OpenACS Project, which is based on the original Tcl version of ACS, using AOLserver and either PostgreSQL or Oracle.

AOLserver's transition to an open-source model wasn't without its problems. While the details are still a bit sketchy, a number of OpenACS developers briefly worked on their own fork of AOLserver, which they called OpenNSD, arguing that the AOLserver developers needed to be more open to community involvement. OpenNSD now appears to be dead, with the OpenACS community once again encouraging people to use AOLserver.

At the same time, ArsDigita added a number of enhancements to AOLserver that have not yet been added to the server's source code in its latest stable or development editions. Because my use of AOLserver is almost always connected to OpenACS, I will be using AOLserver 3.3ad13, available at www.openacs.org/software.adp. More advanced versions, including a development snapshot for the upcoming 4.0 version, are available at www.aolserver.com. But right now, these official versions are not guaranteed to support OpenACS.

Until earlier this year, when version 2.0 was first released to the general public, Apache was a multiprocess server. That is, a number of Apache processes would run at any given time, with each process able to handle a single HTTP transaction at a given time. Supporting ten simultaneous transactions meant having ten Apache processes running in parallel, while supporting 100 simultaneous connections meant having 100 such connections available.

Apache 2.0 changes this somewhat, allowing you to have multiple threads per process. Each thread can handle an HTTP connection, meaning that five threads in each of five processes can accept up to 25 simultaneous connections. Because threads typically consume fewer resources than processes, this effectively means that a typical PC will benefit from an increase in performance.

AOLserver, by contrast, has always been multithreaded and operates within a single process. At any given time, your computer will be running only one copy of nsd, whose name reflects the fact that it was once the NaviServer dæmon. But that one process can handle a large number of simultaneous HTTP connections. Indeed, AOL continues to push AOLserver development precisely because it can handle such a large number of simultaneous connections. It uses AOLserver on its own high-traffic web sites, including netscape.com, aol.com and digitalcity.com.

One advantage—and potential pitfall—of a multithreaded solution is the fact that the threads easily can share data structures with one another. AOLserver takes advantage of this to create a pool of database connections. Because connections in this pool are always kept open, your web applications will not have to spend time opening (or closing) them. Moreover, because it is rare for all of a server's current HTTP connections to require simultaneous access to a database, the pool can contain fewer connections than the maximum number of threads—reducing the amount of memory used by the web server and the database server. You can think of this as the database equivalent of packet switching, in which a telephone line is shared among numerous parties by taking advantage of the fact that no one needs the line 100% of the time.

AOLserver supports plugin modules, much as Apache does. There are a number of modules available, ranging from an XML parser (nsxml) to an embedded version of Python (PyWx). There are also modules for executing CGI programs, for secure connections via SSL and for individual relational databases such as MySQL, PostgreSQL and Oracle. Because OpenACS can work with either PostgreSQL or Oracle, the version of AOLserver available for download from openacs.org includes both of these modules, as well as nsxml.

Just as mod_perl allows web developers to customize Apache's configuration and responses without using C, AOLserver provides an integrated API that allows you to create custom functionality in the Tcl language. Truth be told, I personally would prefer to use Perl or Python for development, but as many AOLserver and OpenACS developers have told me over the years, Tcl “isn't that bad”, and I have managed to create a number of interesting, maintainable applications using Tcl and AOLserver in a remarkably short period of time. (I haven't yet tried the embedded Python module, in part because OpenACS requires the use of Tcl.) And, the API that AOLserver provides makes it relatively easy to work with such things as HTTP headers and HTML form values.

Compiling AOLserver is relatively straightforward. Unlike Apache, which provides support via the apxs program for compiling modules after you have installed the server, AOLserver requires that all modules be compiled and installed together.

While creating this user and group is not mandatory, AOLserver will refuse to run as root for security reasons. So before you begin to compile and install AOLserver, you should create a new user and group on your system, traditionally called nsadmin. On my Red Hat 7.2 system, I simply say:

/usr/sbin/adduser nsadmin

While still logged in as root, I now create the /usr/local/aolserver directory, into which AOLserver is installed by default. I then give ownership of this directory to nsadmin:

mkdir /usr/local/aolserver
chown nsadmin.nsadmin /usr/local/aolserver

su - nsadmin
cd /tmp
tar -zxvf aolserver3.3ad13-oacs1-beta-src.tar.gz
cd aolserver
./conf

The build process can take awhile and doesn't produce a great deal of output to the screen. If you are concerned that the process has somehow become frozen, you can look at the log/aolserver.log file; all of the compilation output can be found there.

When the build process is done, you will have a copy of AOLserver in /usr/local/aolserver. The most important directories under /usr/local/aolserver are bin, in which the AOLserver (nsd) program is located, along with the shared libraries (.so) for each of the modules that were compiled into the server. The log directory contains access and error logs for the server, and the lib directory contains the built-in Tcl interpreter.

AOLserver's configuration file is written in Tcl; a simple configuration file is placed by default in /usr/local/aolserver/sample-config.tcl. If you examine it, you will see that each of the configuration directives is actually a Tcl variable assignment. These variable assignments are divided into sections, where each section normally represents a module that was compiled into the server.

As you can see from the sample configuration file, you can assign literal values to variables. For example, you can set the HTTP port to which AOLserver listens to 8000 with the following:

set httpport 8000

Because the configuration file is written in Tcl, you also can set the homedir variable, which is /usr/local/aolserver by default, by asking AOLserver rather than hard-coding the value:

set homedir          [file dirname [ns_info config]]

set servername             "server1"
set pageroot ${homedir}/servers/${servername}/pages

Other configuration settings are made with the ns_param command, which typically takes two parameters: a name and a value. Each parameter must come in a section, begun by a call to ns_section. For example, we can activate server debugging by turning on the debug parameter in the (global) ns/parameters section:

ns_section "ns/parameters"
ns_param   debug           false

Unfortunately, the documentation for AOLserver's parameters is quite lacking when compared with the on-line Apache documentation. An almost complete list of parameters is at aolserver.com/docs/admin/config-reference.tcl.txt, which demonstrates a server configuration that sets nearly everything.

Once you have finished configuring your system—and the default configuration is a good start for simple sites—you can start AOLserver by invoking the nsd program and specifying the name of the configuration file you want to use:

cd /usr/local/aolserver
bin/nsd -f -t sample-config.tcl

The -f option runs AOLserver in the foreground, sending the error log to your screen. Once you feel comfortable with what's happening, you can remove the -f, looking in the log directory for your server's error log.

If you want AOLserver to listen to port 80, then you must start it as root. Otherwise, Linux will refuse to honor the request, telling you that only the superuser can start servers that listen to “privileged” ports (i.e., less than 1024). If only root can listen to port 80, but AOLserver refuses to run as root, how can you serve port 80? By starting AOLserver as root and passing it options to indicate the user and group to which it should switch:

su root
cd /usr/local/aolserver
bin/nsd -f -u nsadmin -g nsadmin -t \
  sample-config.tcl

You should now be able to point your browser at http://yourhost.yourdomain.com:8000/ and see the introductory AOLserver document, welcoming you to this new server. Note that AOLserver's configuration file looks for both your computer's name and its IP address, so if you are connected to a network, you will not be able to point your browser to localhost, but will instead need to use its full name.

While AOLserver is undoubtedly an excellent HTTP server for static documents, you're unlikely to use it for that. It's far more common to create dynamic pages using Tcl.

The easiest and simplest way is to create a Tcl program that returns HTML. To do this, create a file in the pageroot that ends in the .tcl extension. It can create any Tcl that you want; the most important thing, however, is that it end with ns_return—a Tcl procedure defined by AOLserver that takes three arguments: 1) a numeric HTTP response code (such as 200 or 404) that indicates the success or failure of the program's execution, 2) a “Content-type” header that describes the type of content that is being returned and 3) the actual content to return to the user.

For example, here is a simple “Hello, world” program:

ns_return 200 text/html "<html>
<head>
    <title>Testing</title>
</head>
<body>
     <p>Hello, world</p>
</body>
</html>"

If you stick the above program in the pageroot directory as hello.tcl and load it into your browser, you will get the literal contents of the file returned to you. That's because we need to reconfigure AOLserver to allow .tcl pages within the pageroot. We do this by setting the enabletclpages parameter to true within the ns/server/${servername} section:

ns_section "ns/server/${servername}"
ns_param   enabletclpages  true

A .tcl page can do a host of different things: connecting to a database, retrieving information from XML files to retrieving information from across networks or receiving information from an HTML form. Because Tcl comes with a variety of string-manipulation commands, you can parse the input and interpolate variables into the output in a wide variety of ways.

Note that the Tcl is interpreted within AOLserver, rather than as an external process. This means that such .tcl files execute much faster than a CGI program would; in many ways, they run similarly to Perl programs in mod_perl.

It's true that .tcl files are great when a programmer is creating the outgoing HTML. But as I (and others) have learned from bitter experience, graphic designers generally are unprepared to modify HTML that appears within source code files. For this reason, many web developers have switched over to templates—be they ASP, JSP, HTML::Mason, DTML or a variety of other similar technologies. AOLserver comes with its own built-in templating system, known as ADP (AOLserver Dynamic Pages), whose syntax is suspiciously similar to Microsoft's ASP. Code that you want to execute goes within <% and %>, while the code that outputs a value into the surrounding HTML goes within <%= and %>. For example:

<% set foobar "abcdef" %>
<head>
    <title>Testing</title>
</head>
<body>
    <p>Hello, world</p>
    <p>Hello, <%= $foobar %></p>
</body>
</html>

Sites larger than a few pages probably will want to share some Tcl code. The easiest way to do this is to create one or more .tcl files that define procedures that AOLserver will load at startup time. These procedures then will be available to all of your .tcl and .adp pages. To enable this functionality, we must add the following to our sample-config.tcl file:

ns_section ns/server/${servername}/tcl
ns_param   library \
  ${homedir}/servers/${servername}/tcl
ns_param   autoclose      on
ns_param   debug          true

proc return_hello {} {
    return "hello"
}

<% set foobar "abcdef" %>
<% set hello [return_hello] %>
<head>
    <title>Testing</title>
</head>
<body>
    <p>Hello, world</p>
    <p>Hello, <%= $foobar %></p>
    <p>Hello, <%= $hello %></p>
</body>
</html>

Because you must restart AOLserver in order for it to load library Tcl procedures, I've often found it easiest to define new procedures within <% %> sections in my ADP pages. Once I see that the procedure works correctly, I move its definition to the library directory and restart AOLserver only once.

ADP and .tcl pages are fine for documents that contain some dynamic content. But sometimes you want to associate programs with a URL without necessarily creating a document on disk. We easily can take care of this by defining a Tcl procedure in our library file and then registering that procedure with a particular URL using the AOLserver ns_register_proc command:

proc http_hello {} {
  ns_return 200 text/html "<html>
                           <head><title>Hello!
                             </title></head>
                           <body><p>Hello!
                             </p></body>
                           </html>"
}
ns_register_proc GET /hello http_hello

If you put this Tcl code in a file that sits within the library directory we defined earlier, restart your server and point your browser to /hello, you will see the output from our http_hello procedure.

I've been programming in mod_perl for several years and am still impressed by the ease with which you can create new dynamic pages with AOLserver and ns_register_proc. Moreover, you can register different procedures for GET and POST requests. You even can register filter procedures that can monitor or change the output generated by another page.

If you plan to use OpenACS for on-line communities, then you almost certainly will have to learn how to work with AOLserver. But even if OpenACS does not interest you, AOLserver's flexibility, speed and multithread capabilities are well worth investigating for your dynamic web sites.

Resources