Don't leave your Linux software stuck on one distribution. Make it run anywhere with the standard that all the major distributions use.
The Linux Standard Base (LSB) specifies an interface between an application and a runtime environment. Many distributions have achieved certification for their runtime environments. This article outlines the steps needed to build applications that adhere to the LSB interface.
The LSB Project was founded in 1997 to address the application compatibility problem that was beginning to emerge. Different distributions were using different versions of upstream software and building them with different options enabled. The result was that an application built on one distribution might not run on another distribution. Worse yet, the application often would not work on a different version of the same distribution.
Originally, the LSB was intended to create a common reference implementation for the base of a GNU/Linux system. In addition to the reference implementation, a written specification was to be developed. This idea wasn't well received by many of the distributions that had considerable investments in their own base software, which they perceived as being a competitive advantage.
After further discussion among the interested parties, the LSB Project underwent a fundamental shift in focus in order to achieve consensus among the entire community. The shift gave priority to the written specification over the implementation, and it defined the LSB as a behavioral specification instead of a list of upstream feature/version pairs. This new focus was realized as a three-prong approach: a written specification, which defines the behavior of the system; a formal test suite, which measures an implementation against the specification; and a sample implementation, which provides an example of the specification.
The LSB Specification actually is made up of a generic portion, the gLSB, and an architecture-specific portion, archLSB. The gLSB contains everything that is common across all architectures; we try hard to define as much as possible in the gLSB. The archLSBs contain the things that are unique to each processor architecture, such as the machine instruction set and C library symbol versions.
As much as possible, the LSB builds on existing standards, including the Single UNIX Specification (SUS), which has evolved from POSIX, the System V Interface Definition (SVID) and the System V Application Binary Interface (ABI). The LSB uses the ELF definitions from the ABI and the interface behaviors from the SUS. It adds the formal listing of what interfaces are available in which library as well as the data structures and constants associated with them. See the “Linux Standard Base Libraries” sidebar for the list of libraries currently specified.
In addition to the ABI portion of the LSB, the specification also specifies a set of commands that may be used in scripts associated with the application. It also mandates that the application adhere to the filesystem hierarchy standard (FHS).
One additional component of the LSB is the packaging format. The LSB specifies the package file format to be a subset of the RPM file format. The LSB does not specify that the distribution has to be based on RPM, however, only that it has some way of correctly processing a file in the RPM format.
One final item to mention is the name of the program interpreter. The program interpreter is the first thing executed when an application is started, and it is responsible for loading the rest of the program and shared libraries into the process address space. Traditionally, /lib/ld-linux.so.2 has been used, but the LSB specifies /lib/ld-lsb.so.1 instead on IA32. Generally, /lib/ld-arch-lsb.so.1 is used for other architectures. This provides the operating system with a hook early in the process execution in case something special needs to be done to provide the correct runtime environment to the application. You can pass the following to GCC to change the program interpreter:
-Wl,--dynamic-linker=/lib/ld-lsb.so.1
A long time ago, people realized that code changes are cheaper and easier to make when they come earlier in a development process rather than later. With this in mind, the LSB Project has created a build environment to assist with the creation of LSB-conforming applications. This build environment provides a set of clean headers, stub libraries and a compiler wrapper.
The LSB stores much of its definition in a database. In addition to the portions of the specification that would be tedious to edit manually, we are able to produce a set of clean header files and stub libraries that contain only the things specified by the LSB. Using the database in this way helps to ensure the tools and specification stay in sync as changes and additions are made. The packages you need to install are described in the “Linux Standard Base Packages” sidebar.
The first step in building an LSB-conforming application is to compile the code with the LSB headers. If the code doesn't compile, it probably is using something outside of the LSB. This isn't necessarily a showstopper, but it is something to which you need to pay particular attention. The LSB headers are installed in /opt/lsbdev-base/include. As a quick test, pass -I/opt/lsbdev-base/include to GCC and see what happens. The compiler wrapper described later does this step and some other related steps for you.
Once you have compiled your code, the next step and next test is to link the code together to form the final application. Usually, this step looks like this:
gcc -o app1 obj1.o obj2.o -lfoo
The LSB stub libraries can be found in /opt/lsbdev-base/lib and can be specified by passing the -L option to the compiler. These stub libraries are used only at link time. Typically, the normal system libraries are used at runtime. Again, the compiler wrapper described later handles these details for you.
Once you have linked your application, use the ldd command to see what shared libraries are being used. At this point, there should not be any shared libraries other than the ones specified in the LSB (and listed in the “Linux Standard Base Libraries” sidebar). If there are, you need to take extra steps to make them be linked statically. Usually, the -Wl,-Bstatic and -Wl,-Bdynamic options can be used to specify that certain libraries should be linked statically. By now, you may be seeing a pattern: the compiler wrapper handles this for you.
As an example, here is what the application xpdf typically looks like:
# ldd /usr/bin/xpdf
libXpm.so.4 => /usr/X11R6/lib/libXpm.so.4
libt1.so.1 => /usr/lib/libt1.so.1
libfreetype.so.6 => /usr/lib/libfreetype.so.6
libSM.so.6 => /usr/X11R6/lib/libSM.so.6
libICE.so.6 => /usr/X11R6/lib/libICE.so.6
libX11.so.6 => /usr/X11R6/lib/libX11.so.6
libpaper.so.1 => /usr/lib/libpaper.so.1
libstdc++-libc6.2-2.so.3 =>
/usr/lib/libstdc++-libc6.2-2.so.3
libm.so.6 => /lib/libm.so.6
libc.so.6 => /lib/libc.so.6
/lib/ld-linux.so.2 => /lib/ld-linux.so.2
Here is the LSB-conforming xpdf:
# ldd /opt/lsb-xpdf/bin/xpdf
libSM.so.6 => /usr/X11R6/lib/libSM.so.6
libICE.so.6 => /usr/X11R6/lib/libICE.so.6
libX11.so.6 => /usr/X11R6/lib/libX11.so.6
libm.so.6 => /lib/libm.so.6
libgcc_s.so.1 => /lib/libgcc_s.so.1
libc.so.6 => /lib/libc.so.6
/lib/ld-lsb.so.1 => /lib/ld-lsb.so.1
The non-LSB libraries are not showing up as needed by the application, because they are linked statically into the application itself. There is a trade-off here: the application executable becomes larger, but it has fewer dependencies on the installed operating system.
Finally, we get to the compiler wrapper, lsbcc and lsbc++. These are the same program; they simply are invoked with different names to indicate C or C++ mode. The general idea is you can use lsbcc wherever you would use GCC and lsbc++ wherever you would use g++.
This wrapper tool parses all of the options passed to it and rearranges them slightly. It then inserts a few extra options to cause the LSB-supplied headers and libraries to be used ahead of the normal system libraries. This tool also recognizes non-LSB libraries and forces them to be linked statically.
Because the LSB-supplied headers and libraries are inserted into the head of the search paths, it generally is safe to use things not in the LSB. Make sure, however, that they are not dependent on something that intentionally has been left out of the LSB headers and libraries and that they can be linked statically into the applications. This allows lsbcc to be transparent in most cases.
With the LSB development packages installed, porting a sample application becomes as easy as the normal three-step process, but with a slight difference:
CC=lsbcc ./configure make make install
By telling the configure script to use lsbcc instead of GCC, it conducts its various tests in an LSB environment and configures the software with any adjustments or limitations that may be required. Sometimes this results in a portable replacement for a feature being used. Generally, though, the overall functionality is close to what it would have been if GCC were used instead. As an exercise, try running a configure script both ways and compare the results. Another benefit of telling configure to use lsbcc is that it automatically sets CC to lsbcc in the generated makefiles, so you don't have to remember to pass it in (make CC=lsbcc) every time you run make.
The lsbcc command defaults to calling GCC with the modified arguments, but an environment variable can be used to tell it what compiler to use instead. This should work okay for any other compiler that is command-line option compatible with GCC.
Once the application has been built, use the lsbappchk program to test the program to see if it conforms to the LSB. This program checks the list of shared libraries used by your application; it also checks to make sure you are using only the interfaces permitted by the LSB. Here is an example run:
# /opt/lsbappchk/bin/lsbappchk /bin/ls
/opt/lsbappchk/bin/lsbappchk for LSB Specification 1.3.3
Checking binary /bin/ls
Incorrect program interpreter: /lib/ld-linux.so.2
Header[ 1] PT_INTERP Failed
Found wrong interpreter in .interp section: /lib/ld-linux.so.2
instead of: /lib/ld-lsb.so.1
DT_NEEDED: librt.so.1 is used, but not part of the LSB
Symbol clock_gettime used, but not part of LSB
The LSB does not require that the utilities provided by the OS be LSB-conforming themselves. Therefore, there isn't really an expectation that a distribution's own /bin/ls should pass this test. It simply makes for a handy example.
The output of lsbappchk tells us that /bin/ls is not an LSB-conforming application. The first problem is it wasn't linked with the LSB-defined program interpreter /lib/ld-lsb.so.1. The next problem is that the application is looking for the shared library librt.so.1, which is not included in the set of LSB-defined libraries. Lastly, the function clock_gettime() is used but is not linked statically to the application (it would have been found in librt.so.1).
The general approach to fixing an application such as this would be to rebuild the application using lsbcc, which would set the program interpreter correctly and cause librt.a to be used instead of librt.so. Sometimes, statically linking a library can cause new non-LSB symbols to be brought into the application, so this process may have to be repeated a couple of times.
In some larger applications or in sets of related applications, it may be desirable to create shared libraries that are used only by these applications. This is permissible under the LSB as long as the shared library is installed as part of the application and it resides in the application private data area, not in any of the system library locations. The -L option to lsbappchk lets you tell the testing tool the full path to the shared library, which is considered to be a part of the application for the purpose of testing conformance to the LSB. Here is an example of an LSB-conforming build of the Apache Web server, which uses three private shared libraries:
# /opt/lsbappchk/bin/lsbappchk \ -L /opt/lsb-apache/lib/libaprutil.so.0 \ -L /opt/lsb-apache/lib/libexpat.so.0 \ -L /opt/lsb-apache/lib/libapr.so.0 \ /opt/lsb-apache/sbin/httpd /opt/lsbappchk/bin/lsbappchk for LSB Specification 1.3.3 Adding symbols for library /opt/lsb-apache/lib/libaprutil.so.0 Adding symbols for library /opt/lsb-apache/lib/libexpat.so.0 Adding symbols for library /opt/lsb-apache/lib/libapr.so.0 Checking binary /opt/lsb-apache/sbin/httpd
As I mentioned earlier, the LSB specifies that a package must be delivered in the RPM file format. This does not mean that RPM has to be used to build or package your application, although it may be the most practical option, depending on whether you already are using it. Other options would be creating the package in the Debian format, and then using alien to convert it to RPM. Or, you could use some other tool for creating the RPM file format. We have the beginnings of a tool called mkpkg to create the RPM format file, but it likely will require something to sit on top of it to make it useful to any but the most die-hard hacker.
In our application battery, we currently build the application and install it in a temporary root and then invoke RPM to package up the install application. This may seem a little clunky, but it works without much pain and produces more consistent results across all of the different versions of RPM found in the wild.
Here is a sample spec file for the xpaint application:
Summary: An X Window System paint program Summary: XPaint Name: lsb-xpaint Version: 2.6.2 Release: 3 Vendor: Free Standards Group License: MIT Group: Appbat/graphics Buildroot: /usr/src/appbat/pkgroot/lsb-xpaint AutoReqProv: no PreReq: lsb >= 1.3 %description LSB conforming version of xpaint. XPaint is an X Window System color image editing program and painting program. Xpaint is added to the LSB Application Battery primarily to demonstrate the use of X11 libraries. %pre %install %post %preun %postun %clean %files %attr ( - bin bin ) /opt/lsb-xpaint
Full source code for building and packaging this and the other applications in the application battery can be found in the LSB Project CVS tree.
Yes, it really does work, although to be fair, we still are running into corner cases and various applications that don't always follow the rules for clean, portable code. As part of the verification for the LSB, we have created an application battery built from the tools described here. This set of applications includes Apache, Samba, Lynx, Python, xpdf and groff. We have tried to select a set of real applications that provide coverage over as much of the LSB set of interfaces as possible.
LSB version 1.3 does not support C++, so the rule requiring the library to be linked statically applies. We are adding support for C++ to LSB 2.0 to avoid this. We provide the lsbdev-c++ package, which contains a version of libstdc++ that was configured and built with lsbcc. This and GCC version 3.2 seem to produce good results. We have tried other combinations of compilers and different versions of the C and C++ libraries but ran into various problems, depending on the nature of the application.
For the LSB in general, we will continue to add additional libraries to the specification as long as there is consensus that they are needed and have reached a certain level of stability. This should help close the gap between how distribution-provided and LSB-conforming applications are built.
For the LSB development environment, we will continue to make the tools better and more transparent. The development environment is being maintained actively, and feedback from people using these tools is appreciated. With the addition of C++ in LSB 2.0, the development environment will be able to drop the lsbdev-c++ package being used today in favor of the C++ stub library, which will move into the base LSB development package.
Currently, you may have to set several options in an rpmrc or rpmmacros file to make RPM produce LSB-conforming packages. It is our hope that we can come up with an LSB mode for rpmbuild that can handle all of this automatically. Hopefully, it will make it even easier to build existing packages that conform to the LSB.
First off, thanks to the Free Standards Group and its members for providing the support to the LSB Project that has enabled us to accomplish as much as we have. Secondly, thanks to the core group of developers working on the development environment for the LSB, including Chris Yeoh, Marvin Heffler and especially Mats Wichmann for their patience and persistence during the more experimental phases of this project.
Resources for this article: /article/7459.