Perl Embedding

This article describes my experience embedding Perl into an existing application. The application chosen, sc, is a public-domain, character-based spreadsheet that often comes as part of a Linux distribution. The reason for choosing sc was twofold. First, I use sc for any spreadsheet-type tasks I have. Second, I was somewhat familiar with the source, because I once added code in order to format dates the way I wanted. Besides, I always thought it would be nice if sc had some sort of macro language—everything should be programmable in some way.

The first thing I did was to get the sc 6.21 source and compile it on my machine. This ensured that everything worked from the start, before I started making modifications to the code.

The next thing was to add the necessary code to sc.c to embed the Perl interpreter. The basics of this were:

Nothing else needs to be done; the Perl interpreter is now in the code. Obviously you can't do anything yet, but you can work out any compilation problems. Right away, I had a few problems with some #define statements and a prototype for main. EXT and IF were the two offending #defines. I fixed these by appending “sc” to the end of them wherever they occurred in the original sc code, to make them unique. If you were writing an application from scratch, it would not be a bad idea to prepend a common prefix to each #define.

Perl, on the other hand, expected main to have a third argument, env, so I added it. I am still not sure where this argument comes from, but it doesn't seem to create any problems.

Once the base interpreter compiled successfully, I needed a way to call the functions. I looked at the sc source and found that one of the keystrokes, ctrl-k, was free for my use. I used this as my “call Perl” key-command macro, with macros from 0 to 9 defined. This combination calls predefined Perl subroutines called sc_macro_[0-9], when defined. The code in Listing 2 adds this functionality.

The function call_sc_perl_macro checks first to see if the subroutine exists with perl_get_cv. If null is not returned, it calls the function which has the name sc_macro_# where # is a digit from 0 to 9.

The perl_call_va function comes from Sriram Srinivasan's book Advanced Perl Programming, published by O'Reilly. This code was used to expedite my ultimate goal of embedding Perl into sc. The code for perl_call_va can be found in the file ezembed.c.

With sc compiled, I proceeded to test the interpreter by creating dummy macros in the file sc.pl to write some data to temporary files. Everything worked fine, which told me the Perl interpreter was working inside of sc.

With a working Perl interpreter embedded into sc and the ability to call Perl “macros”, the interfaces to the C functions in sc needed to be created to do useful work. Fortunately, sc is laid out nicely enough that, for the most part, all one has to do is wrap an already existing function and interface with its internal command parser.

The first thing I thought might be useful is to move the current cell around. Without that ability, I would be able to operate only on a single cell, which is not very useful. Besides, it was one of the least complicated sections of code and provided a good start.

The code for sc_forward_row is shown in Listing 3 and found in sc_perl.c. Before I describe this code, let me give you a quick overview of how Perl treats scalars. Each scalar has many pieces of information, including a double numeric value, a string value and flags to indicate which parts are valid. For our purposes, the scalar can hold three types of values: an integer value (IV), a double value (NV) and a string value (PV). For any scalar value (SV), you can get to their respective values with the macros SvIV, SvNV and SvPV.

Now, in the Listing 3 code, XS is a Perl macro that defines the function. dXSARGS sets up some stuff for the rest of XSub, such as the variable items that contains the number of items passed to Xsub on the Perl argument stack. If the argument count does not equal 1, XS_RETURN_IV returns 1 to Perl to indicate an error. Otherwise, the top element of the Perl argument stack, ST(0), is converted to an integer value and passed to the forwrow function.

Note that all of the XSub code was generated by hand. Some of this work can be done with Perl's xsubpp or with a tool called swig, but in this case, I felt it was simpler to code it myself.

Finally, tell the Perl interpreter about this Xsub with the statement:

newXS("sc_forward_row",sc_forward_row,"sc_perl.c");

The first argument is the name of the subroutine in Perl. The next argument is the actual C routine (in this case they are the same, but they don't have to be). The last argument is the file in which the subroutine is defined, and is used for error messages. I chose to create all of the newXS functions by parsing my sc_perl.c file with a Perl script, so that I would not have to do two things every time I added a new XSub.

The next thing I wanted was the ability to dynamically load other Perl extensions such as the Tk extension, database extensions or anything else that might prove useful. This requires an xs_init function in place of the NULL in perl_parse as shown below.

perl_parse(MyPerl, xs_init, 2, my_argv, env);

To create the xs_init, I used the following code:

perl -MExtUtils::Embed -e xsinit -- -o - >xs_init.c

One of the first problems I ran into was the fact that Perl redefined yypars to be Perl_yypars. I fixed this by putting new #define statements around places where I used sc's yypars. This created a lot of compiler warnings, but did allow the code to compile correctly.

The other problem I encountered was with the SvIOK and SvNOK macros. These check an SV for a number or an integer, or more precisely, they check to see if the double-value portion of an SV is valid at that point in the code.

Originally, I had the SvIOK and SvNOK macros around any code to which I was expecting to send an integer. The problem is this will not accept code like the following,

sc_put_num_val("34.3"); # this is in perl

because it is being passed a string value and the number part of the SV was not valid at that time. The SvIV and SvNV macros will convert this to a number even if it is not a valid string. I was parsing strings from a file using regular expressions, and the value I would get in $1 would be a string, even though it was numeric. Once I realized SvNV would produce a number for me, my test script started working.

This example is not the cleanest implementation of embedding Perl into an application. It was meant as a quick solution to a problem. With an embedded Perl interpreter, sc is quite a bit more powerful than before. One example included with the source is a mortgage calculator that grabs the interest rates form the CNN Financial News web site. With all the Perl modules available, the possibilities are endless.

Resources