There is more to the Internet than sending JPGs. See how Linux and the MBone addresses the needs of distance learning.
One of the most promising applications of the Internet is distance education. The most significant advantage of Internet connectivity over previous communication paradigms for distance education is that a high degree of interaction among teachers and learners is possible. Education is a highly interactive process. If it weren't, schools wouldn't be needed—textbooks alone would be sufficient. The fact that the Internet protocols support two-way communication and virtually unlimited possibilities for integrated multimedia (given sufficient bandwidth and quality of service) presents unprecedented opportunities for extending the reach of education beyond the traditional classroom.
Most applications of Internet technology to distance education have been web-based courses. In a web-based course, the course content is developed as modules in HTML, and interaction takes place asynchronously through discussion boards or plain old e-mail. Some web-based courses also support synchronous interaction through chat sessions. Some advantages of web-based courses are that they work well at modem bandwidths and require only a web browser on the student's computer, thus maximizing the possible audience and minimizing the technical support requirements. However, a quality web-based course requires a great deal of effort to produce; and in disciplines such as engineering where the content must be updated frequently, maintenance of the course can also be significant. Another important limitation is that interactions are essentially limited to textual media. In answering a question for a student in an engineering class, the teacher often needs to construct or mark up a diagram while explaining the concept. This is best accomplished in an environment which supports real-time audiovisual interaction, which is why the traditional face-to-face classroom has survived the last few hundred years so well.
With this in mind, the “MBone virtual classroom” was developed at North Carolina State University to allow students to attend live engineering classes from a remote location by “tuning in” to the class from a workstation. The concept was to replicate as nearly as possible the face-to-face environment with real-time interaction including audio, video and graphics. To solve the problem of having many remote students attending the classroom virtually, IP-multicast and the MBone tools were employed. The virtual classroom has been in use since the fall of 1996 to provide access to engineering classes to students at several locations in North Carolina. This article describes how IP-multicast and the MBone tools were used to create the virtual classroom environment, and the development of DETA, the Distance Education Teaching Assistant, a Tcl/Tk-based wrapper which provides a simple, unified interface to the many hardware and software components of the virtual classroom.
IP-Multicast, the class-D addressing scheme in IP which makes the MBone network possible, was developed by Steve Deering in his PhD thesis at Stanford, and later developed and implemented at Xerox Palo Alto Research Center (PARC). The first multicast tunnel was established between BBN and Stanford University in the summer of 1988. The Internet Multicast Backbone (MBone) was subsequently established as a virtual network of multicast tunnels over the existing Internet. In 1992, the same year that the Internet grew to one million hosts and Mosaic was created at NCSA, the MBone carried its first real-time audio and video traffic.
IP multicast is useful in that it provides an efficient mechanism for “broadcasting” data over the Internet. It is best understood in comparison to IP unicast. When sending the same data to multiple clients using unicast, multiple separate connections to those clients must be opened. When the number of clients grows significantly, the load on the sender increases dramatically. With IP multicast, the data needs to be sent only once. The multicast-enabled network sends copies of the data to all of the clients who wish to receive the data. In this way, the sender transmits the data only once, regardless of the number of clients wishing to receive the data. It is very similar to a television broadcast, where a single transmitter sends out a single video transmission, and anybody within range of the signal can receive the transmission. Because it is more efficient at sending the same data to multiple recipients, multicast is ideal for multimedia network applications such as video-conferencing or live netcasts.
The ability to send and receive IP multicast is primarily dependent on your network. The network's routers must know how to deal with multicast packets. A few years ago, there existed very few routers capable of handling multicast packets. At that time, a method was needed to send multicast packets over networks designed to handle only unicast packets. This method became the Virtual Multicast Backbone, or MBone. Software that uses the MBone essentially packages multicast packets inside of unicast packets, which non-multicast-enabled routers know how to handle. Multicast-enabled routers are able to identify and process the multicast packets, as well as computers running MBone software.
The primary components of the distance education software used at NC State are the MBone tools. These are vic, rat, wbd, and sdr are available for download in binary and source form at University College London's (UCL) web site at http://www-mice.cs.ucl.ac.uk/multimedia/software/. vic is an MBone video-conferencing tool. It was originally developed by Steve McCanne and Van Jacobson at the Lawrence Berkeley National Laboratory (LBNL) Network Research Group. The version being used at NC State is currently under development at UCL. This version provides video-capture support using Video4Linux, so many existing video-capture cards are compatible with it. It incorporates a number of codecs including H.261 and H.263. It provides controls to adjust frame rate, bandwidth, and video quality, as well as many other options. Users can switch between thumbnail and full-screen video windows, and switch between a number of video formats. vic runs in multicast- conference mode or point-to-point-unicast mode.
The Robust Audio Tool (rat) is an MBone audio-conferencing tool. rat was developed by UCL's Networked Multimedia Research Group. There are two versions of RAT: a stable, toll-quality (i.e., telephone-quality) version 3, and an improved, though experimental, high-quality version 4. rat supports both ALSA (Advanced Linux Sound Architecture) and OSS (Open Sound System), so it is compatible with a large number of sound cards. rat provides a number of audio codecs, as well as packet-loss concealment schemes. Other features include automatic gain control, silence suppression and encryption. rat also provides a graphical interface showing conference participants and audio levels. Like vic, rat can operate in point-to-point-unicast mode or in multicast-conference mode.
wbd is an MBone shared whiteboard. It allows a number of participants in a conference to share a single whiteboard workspace. It was originally written by Julian Highfield at Loughborough University. The most recent development work on it has been done by Kristian Hasler at UCL. wbd is compatible with the original LBNL whiteboard, wb, developed by Steve McCanne. Because wb is available only in binary form for UNIX platforms, Julian Highfield wrote wbd primarily to fill the need for a Windows version of wb. Since the source is freely available, we have chosen to use wbd over the binary-only wb on Linux. wbd has a standard set of whiteboard features, such as font, color and line-width options, text input capability, drawing tools and various page orientations. wbd can import both PostScript and text files. wbd was designed to work in both point-to-point and shared multicast modes, though currently only the multicast mode functions properly in Linux.
Instead of each user in a conference connecting to every other user, MBone users join a multicast group. Anything sent to the group is received by all current members of the group. None of the MBone tools discussed so far provide any means of locating or advertising these groups. This is accomplished through the session directory tool, sdr. In a sense, the session directory is like a television guide which shows all the currently available “shows” on the MBone. sdr was originally written at UCL and was modeled after another LBNL tool called sd. When the user loads up sdr, a listing of all public and private MBone sessions appears. To get more information about a specific session, the user clicks the session name. To join a session the user clicks the join button for that session and sdr then loads up the various tools needed to participate in the session. To create a new session, the user clicks the New button and enters various information about the session. sdr then generates the multicast addresses and advertises the session for other users to see.
The bandwidth requirements for the MBone tools are relatively low by current standards. Each video source requires only about 128KBps at a rate of ten frames per second. The audio requires about 64KBps at telephone quality. Higher frame-rate video is possible, but we've found that high-quality whiteboard data in combination with good-quality audio more than compensate for the slow video. The video mainly orients the participant and provides visual cues, while the actual content is provided via the audio and whiteboard data. For some classes, it is more important to provide full-motion video, and when adjusted appropriately, vic can provide this.
The MBone tools were not specifically designed with distance education in mind. In fact, though interoperability and uniformity between the applications was a goal of the original designers, it had yet to be achieved when the first MBone classes began at NC State. Additionally, the tools themselves have a relatively steep learning curve for nontechnical users. For example, in order to create a new session, a number of technical options have to be set in sdr, including Type of Session, Session Scope, Type of Media and Session Time. Experienced users have little difficulty navigating these options, but to meet the needs of distance education, users have to be able to operate the tools with only minimal training. In order to accomplish this goal, the distance education Teaching Assistant was created. DETA is a wizard-style interface to the MBone tools. It hides all of the technical options the inexperienced user should not have to deal with. Originally, it primarily functioned as a simplified interface for starting Mbone-based Distance Education classes. Over the years, a number of features have been added, including the ability to give preprepared presentations and automatic digital archiving of classes.
The underlying purpose behind DETA is to meld the various tools and supporting programs into a single, easy-to-use application. DETA is written in the Tcl/Tk interpreted scripting language. The primary reason for this choice has to do with the MBone tools. All of them are Tcl-based applications written in C or C++. This not only provides great flexibility within the applications themselves, but also allows for basic interoperability between them. This interoperability is achieved through Tcl's Send command. Send allows one Tcl interpreter to execute commands remotely within another applications interpreter. In DETA, the Send command is primarily used to remotely operate SDR, as well as to set options within the other MBone tools.
One of the drawbacks of using the Tcl Send command for communication is that it doesn't hide any of the various tool's internal implementation. This becomes an issue when that implementation is changed, or a new tool is added. The core DETA script then has to be altered to adapt to the changes in the tools. One solution to this problem is to modify the various tools directly so that they communicate via a common, implementation-independent bus. This would require separate modified versions of the MBone tools. Rather than use modified versions of the MBone tools, it was decided to modularize the code that communicates with the various tools into separate interface packages. The interfaces contain all of the tool-specific Tcl code in string arrays. These arrays are sourced by the primary DETA script at startup. In this way, the core script can be modified without affecting implementation-dependent code, and new or updated tools can be added simply by creating new interface modules.
Figure 1. Mode Selection Screen for DETA
When the user first starts DETA, he or she is given four mode options, as seen in Figure 1. The first is to create a new session. This creates a new multicast session that will be announced while the session is active, and which will expire after the session becomes inactive. By creating the session, the user becomes the designated host of that session. The second option is to host an existing session. This mode is included so that users can host permanently existing sessions. At NC State, a session directory server was written by Troy Holder that continuously announces class sessions. In this way, semester-long class announcements do not have to be re-created at every meeting. The third option is to join an existing session. This is the option that a remote participant would select. It allows the user to select a desired session from a list of available sessions and then join it. The last option is to play a recorded session. This mode is used to select a previously archived session for playback via a remote VCR server. Once the mode is selected, the user enters information about the session. Figure 2 shows the information required when creating a new session. This includes the name of the session, the lecturer name, and the MBone tools to use in the session. This is saved between sessions and usually doesn't need to be re-entered. Once the options for the session have been set, DETA creates the session and loads the MBone tools.
Figure 2. User Information Screen for DETA
In addition to melding the core MBone tools, a number of features have been added to DETA. The first is a tool called WBDImport developed at NC State. This tool is modeled after WBImport, which was written by Van Jacobson. It allows the user to give a PowerPoint-style presentation using the shared whiteboard tool. The format used for slide files is generic PostScript. The user indicates which files will be used at startup, either by specifying a directory containing the slides, or by specifying a text file containing a list of all the files and their locations. During the session, a WBDImport window lists all of the slides specified at startup. To show a slide, the user simply clicks the name of the desired slide and it is loaded into the shared whiteboard. This tool allows professors to prepare a set of slides in advance and then annotate them during a lecture. The slides and the annotations are sent out live to all the participants in the session. Using preprepared slides in this way has been found superior to handwriting for classes requiring large amounts of written notes.
The audio tool used in DETA allows only one user to speak at a time. If there is a question at a remote site, it is important that there be someway to signal the host so that the question can be asked. A tool called the Electronic Hand Raiser provides this capability. Remote users have a control panel with a button labeled “Ask Question”. When they click this, the session host will hear a tone and see a message button indicating who the question is from. To acknowledge the question, the host clicks the message, and then allows the remote site to ask their question. There is also a “Cancel Question” button at the remote site if the remote user wishes to withdraw their question. In addition to dealing with the problem of half-duplex audio, this tool provides a form of floor control that matches traditional classroom protocols (See Figure 4). Figure 3 shows a live DETA session with the MBone video, audio and whiteboard tools, as well as the Electronic Hand Raiser and WBDImport tool.
Figure 3. DETA in “host” Mode
Figure 4. DETA Playing Back a Session
Another feature of DETA is integrated recording and playback of sessions. To record a session, the host checks the record option and the session is automatically recorded on a separate VCR server. To play back a session, the user selects the play session option and then chooses the desired session. Once the session is selected, DETA provides a VCR-style interface (see Figure 4). This interface also allows the user to start the various tools and provides standard VCR controls such as play and rewind. The VCR server itself is a separate Tcl script that DETA communicates with via TCP/IP sockets. The primary function of the server is to start and stop a Java application called mVCR. mVCR was written by Peter Parnes at the Lulea University of Technology in Sweden. It is capable of capturing and playing back multicast data streams for the MBone audio, video and whiteboard tools. The DETA VCR Server and mVCR, in combination with DETA, provide immediate on-demand playback of sessions. As soon as a class session is complete, that session is available for remote playback. This greatly reduces the cost and overhead necessary to provide time-delayed classes to remote sites. Additionally, students who missed a class or wish to review a class can individually replay recorded sessions.
The recording and playback of DETA sessions is not entirely without problems. The recording system works by capturing all the network packets received at a client station during the initial live session, encoding them along with a time stamp and writing them to disk. On playback the time stamps are used to deliver an “identical” stream of network packets that will exactly mimic the original session and preserve the synchronization of the audio, video and whiteboard. The quality of the recording (or the live session) can be affected by packet loss due to network congestion, processor loading, or other sources. Generally speaking, the loss of a single packet does only minimal damage to the audio or video recording. A missed syllable during playback is easily filled in from the context, and a missed video frame causes only a momentary pause or distortion. This is often not the case for the whiteboard. One of the ways faculty use the whiteboard is to load preprepared PostScript slides containing figures or equations, and then use the mark-up capability of the whiteboard to provide additional information and lead the class through the discussion of the presented material. A typical PostScript image, even when compressed, may be 10 to 50KB. In order to stream this to the client systems, the image is broken into multiple network packets by the server and reassembled at the client prior to rendering. If even a single packet is lost in transit, the reassembled image will be incomplete and the rendering will fail. The entire image will be absent from the client whiteboard. The whiteboard protocol contains a feature for dealing with this problem during a live session. Each network packet is numbered so every client can identify when a packet is missing. That client can then send a request to all the other session participants asking for the packet to be retransmitted, and any client that has the requested packet can respond. This late delivery mechanism works well for the whiteboard because it less sensitive to the synchronization. Unfortunately, the mVCR recording tool is passive. It simply records the packets that are received without requesting retransmission of missing packets. Whenever a PostScript image is received with any packets missing, it will be incomplete in the recording, and will not appear during playback. This can significantly degrade the quality of the learning experience, since the students viewing the playback are unable to see the prepared material.
To overcome this difficulty, a repair tool was developed: fix_wb_recording.pl. The Perl tool simply parses through the whiteboard recording, locates any PostScript images with missing data, and replaces them with complete images. Parsing the recording requires detailed knowledge of both the mVCR recording format and the wb data format. Neither of these is well documented but sufficient information is available on the Web. The mVCR recording format is described by Peter Parnes at www.cdt.luth.se/~peppar/progs/mMOD/doc/fileformat.txt. The mVCR format is just a header containing basic setup information (multicast address and port number) and any number of wrapped wb data packets. The wb data format has never been published but Lars Rasmusson has posted a reverse-engineered description of the format at www.it.kth.se/~d90-lra/wb-proto.html. While it is known that this description is incomplete and incorrect in some details, it was sufficiently accurate for the purpose of repairing the mVCR recordings. Within the wb data format, PostScript images appear as a sequence (not necessarily consecutive) of draw commands that encode the PostScript data and issue the command to the client to render the complete image.
Whenever the parsing routine encounters a PostScript image in the recording, it opens each PostScript file that was used in the original session and searches for a matching data block. Since most instructors prepare all of their PostScript images using a single software tool, the images often contain large sections that are identical. Therefore it is usually necessary to check several recorded data blocks before one is found that produces a unique match with one of the original images. Once a unique match is found, each draw command associated with the recorded image is deleted and new draw commands are inserted containing all of the data blocks of the complete image and an associated render image command. After a recording has been completed, DETA gives the user the option of post-processing the slides with the repair tool. If this option is chosen, DETA sends the actual PostScript slides to the VCR server and runs the fix_wb_recording.pl tool. Once this was implemented, problems with the quality of the recorded sessions dropped to nearly zero.
Figure 5. The MBone Classroom at NC State
North Carolina State University has been running MBone classes since the fall of 1996. In this time, a number of undergraduate engineering courses have been delivered to participating universities and community colleges. At NC State, a classroom was constructed specifically for distance education (see Figure 5). This classroom contains seating for local students. There are two remote-control video cameras, one for the lecturer and one for the students. There are three large television monitors on which the computer screen is shown to the local students. There is a control area with a computer workstation, document camera, two small TV monitors and an AMX controller. The AMX controller controls the cameras as well as the other audio and video sources. It also provides a central control to operate the other equipment in the room. The most innovative feature of the classroom is a digital projection SMARTBoard mounted on the front wall of the classroom. The SMARTBoard is an input device manufactured by SMART Technologies. It is essentially a touch-sensitive version of a standard office whiteboard. Whatever is written on the surface of the SMARTBoard is transmitted to the computer. The projection version used in the classroom works in combination with an LCD projector to project an image of the computer screen directly onto the SMARTBoard. In this way, the lecturer essentially writes directly on the computer screen. The SMARTBoard is large enough that the local students see directly what the lecturer is writing on the SMARTBoard. This technology provides a natural closed-loop interface to the computer that closely parallels the traditional classroom blackboard.
Generally, a teaching assistant operates the computer equipment and cameras while the instructor lectures. This frees the instructor from having to deal with any technical issues while the class is in session. One of these issues relates to the floor-control aspect of DETA. The floor-control provided by the Electronic Hand Raiser is purely voluntary, and requires all participants to follow the correct protocol. Often we have found that remote participants will scribble onto the shared whiteboard to get the lecturer's attention, or the instructor will simply fail to acknowledge incoming questions. A good solution to the floor-control issue remains to be found and usually it is the assistant's responsibility to help the lecturer acknowledge any questions. Another associated issue occurs when a local student asks a question. The lecturer must repeat the question in order for it to be heard by the remote sites. Often the lecturer will mysteriously stop speaking for a moment, and then start answering a question that none of the remote participants ever heard asked. Remote participants are then forced to either figure out what had been asked, or interrupt the flow of class and ask the lecturer what the original question was. One solution to this problem has been to give students their own microphones. Unfortunately, this relies either on them remembering to activate the microphone, or on distracting continuous presence audio.
The overall response by students to the MBone classes has been positive. The interactive capabilities provided by the MBone tools are far superior to videotapes or broadcasts. They provide a richer educational experience, more similar to a traditional classroom. Most instructors have been able to adapt well to the technology, though there exists somewhat of a learning curve for those accustomed to traditional classroom teaching. One of our primary aims in the future is to flatten this learning curve and make the technology more transparent to the user. Ideally, an instructor should be able to walk into a classroom, activate the equipment, and begin lecturing immediately, without giving any more thought to the underlying technology. While this goal has yet to be achieved, we feel that the MBone tools and DETA, in combination with the Linux platform, represent a highly usable and cost-effective vehicle for the delivery of interactive distance education. For more information, as well as links to all of the DETA and MBone software, visit our web site at http://www.engr.ncsu.edu/deta/.
