An overview of the LSB, the state of LSB specifications, tools, compliance process and lessons learned.
Linux is a dynamic and exciting platform, and its presence in the marketplace is now an undeniable fact. But, to an outsider, it can be tempting to ask “which Linux?” Given the several leading implementations and several variations throughout the world, it's easy to wonder how the platform maintains any consistency.
This is the problem that motivated the creation of the Linux Standard Base (LSB). By reducing the differences between individual Linux distributions from the application developers' point of view, the LSB greatly reduces the costs involved with porting applications to different distributions, as well as lowers the cost and effort involved in after-market support of those applications.
For many independent software vendors (ISVs), supporting Linux also becomes a question of “which one?” Either they choose a particular Linux vendor/release and build, test and support that one (or several). Or, they build against an older distribution, hoping to pick up only “stable” interfaces, shipping their own builds of known problem libraries and wish for the best. By defining a known, consistent set of interfaces, the LSB gives ISVs a better chance of building an application that will work across multiple distributions.
The LSB consists of the following key components:
Specification: a set of requirements that compliant applications and distributions must meet.
Tools: a set of tools, tests and informational systems that help develop the LSB in a collaborative manner as well as help automate the certification process for applications and distributions.
Certification: a process to ensure the two-way promise of platform/application compatibility using the tools and the specification.
The LSB is a core standard for the Linux operating system that encourages interoperability between applications and the platform. Figure 1 shows the components and processes of the LSB.
Figure 1. LSB Components and Processes
The LSB specification defines a common set of elements across multiple Linux distributions that covers the following:
Packaging and installing guidelines.
Shared libraries and their interfaces.
Configuration files.
File placement (Filesystem Hierarchy Standard = FHS).
System commands.
Because it is a binary specification, the LSB is divided into both general and processor-specific components. The following computing architectures now are supported:
x86 (IA32).
x86-64 (AMD64/EM64T).
Intel IA64 (Itanium).
IBM PPC 32.
IBM PPC 64.
IBM 31-bit S/390.
IBM 64-bit zSeries.
By providing a number of guarantees about a typical Linux distribution, LSB eases the burden for ISVs wanting to write applications for Linux and support users on their preferred distributions.
The LSB workgroup provides a number of tools to support the specification. The most important ones are discussed below, and Figure 2 presents a general overview of the tools.
Figure 2. LSB Tools
The LSB SDK allows application developers to build their applications in a special environment to ensure that all external dependencies of the resulting binaries meet the LSB requirements. The LSB SDK includes special header files that contain only LSB-defined functions and data types, stub libraries that export only LSB-defined symbols, and compiler wrappers lsbcc and lsbc++ that launch underlying GCC components with proper settings to use the special headers and stub libraries. Using the LSB SDK is easy—one simply should replace calls to GCC with calls to the compiler wrappers (for example, by changing CC/CCX variables). Special .pc files are provided for those developers using pkg-config. Also, integration of the SDK with Eclipse is supported via the LSB Eclipse Plugin, which adds additional project types (LSB executable, LSB shared and static libraries) to the Eclipse environment and allows one to manage all the settings in a visual way.
The Linux App Checker is used by ISVs and application developers to check their applications for portability across multiple Linux distributions. The tool is not limited to checking just LSB requirements; it also enables general portability analysis based on an internal knowledge base of distribution contents. Currently, the knowledge base contains information about 70 different distribution versions. App Checker accepts as input a set of components comprising an application's package: binaries and .sos, possibly in various directories or/and packed in .rpm/tar.gz (arbitrary mix) files. Results of the analysis are visualized in interlinked HTML reports. For example, external dependencies (libraries and interfaces) of the application as a whole (internal dependencies between components are excluded) are visualized with the info of the “portability degree” for each dependency and recommendations on how to improve portability. An interesting aspect is that the tool can differentiate between required libraries (registered as DT_NEEDED records in ELF) and libraries actually used by the application.
The LSB Distribution Checker is a framework for distribution developers and OSVs (operating system vendors) to check their systems' conformance to the LSB. It is actually a whole stack of software components. The bottom layer consists of actual tests that check specific requirements for particular components in the target system under test. The majority of these tests are runtime unit tests, but there also are special checkers, including static verification mechanisms. The middle layer is the framework to control test execution and results collection. And finally, the top layer is the user interface over all the components. The browser-based interface supports “one button” execution of all certifications tests. Alternatively, the user can select specific tests to run, which can be useful for upstream developers interested in testing only particular components. Note that the LSB Distribution Checker can be used not only for LSB compliance testing, but also as a general QA testing framework. During development of the tests and their regular execution, the LSB workgroup has identified hundreds of bugs in various distributions and upstream components, which were reported and acknowledged by the upstream and distribution developers.
The LSB Navigator is an integrated on-line informational system over a corresponding database that holds structured information about LSB elements, about the Linux ecosystem, as well as service data used in the standardization analysis and decision making. Correspondingly, one can distinguish three parts of the Navigator:
1) Interactive on-line version of the LSB specification, which might be much more useful in a practical perspective compared to the plain specification text. It provides searchable and browse-able information about standardized and non-standardized Linux modules, libraries, interfaces, types and so on. For example, given an interface name, one can find out the following in two clicks:
Whether the interface is in LSB.
Recommendations on using the interface or any of its alternatives.
Direct links to the interface's documentation.
What libraries in which distributions provide the interface.
How many registered applications use the interface.
Which open-source tests check the interface (can be used as interface usage examples).
2) Analytical data about popular Linux distributions and applications, for example:
Interface “popularity” lists—how many applications use each interface (separately for LSB and non-LSB interfaces).
See what external libraries and interfaces are required and actually used by popular applications.
See which elements modern Linux distributions consist of (particular versions on various hardware platforms).
Analyze and compare distributions (statistics on Linux elements in each distribution, which distributions provide/miss a particular version of a component, library, command, interface and so on).
3) LSB workgroup services for structuring standardization process and decision support, for example:
Statistics on LSB evolution (which element appeared/disappeared in which LSB version).
Analytical data on which popular elements (used by numerous applications and provided by majority of distributions) are not yet in LSB—this serves as a basis for expanding the standard.
Service data like the test coverage and documentation coverage of particular libraries.
All the tools are integrated—there are links in the Application and Distribution checkers to pages in the Navigator. Also, the checkers are integrated with the Certification system and allow one to submit test data to the Linux Foundation for certification, right from within the checkers.
LSB certification is a two-way promise of platform/application compatibility. For distributions, being certified is a guarantee that all required LSB commands and interfaces are available on the platform. For ISVs, it is a guarantee that their applications do not rely on anything not provided by either the LSB or the ISV itself in its application package or dependent packages. Both distributions and applications can be certified.
The vendor submits the test results to the Certification Authority (the Linux Foundation). The Linux Foundation reviews the results and either issues the certification or requests the vendor to correct issues and/or resubmit tests with questionable results. The vendor will need to sign a Trademark License Agreement to use the LSB trademark and pay any necessary fees.
The best way to get started with certification is to use the checkers—the LSB Distribution Checker for OSVs and the Linux Application Checker for ISVs. Once the vendor has addressed any issues found by the tools, the checkers can upload the test data to the Certification system to begin the certification process.
Certification puts the application or distribution on the Linux Foundation's official certification list, and it provides marketing materials for developers to advertise their certification. You also will be allowed to use the LSB trademark/branding in your packaging/advertising.
LSB certification secures a number of benefits for the whole Linux ecosystem:
App vendors need to do only one port.
Distro vendors get more apps ported to the Linux platform.
End users can choose their distro and their apps without vendor lock-in .
There is an auto-catalytic effect driven by the LSB (Figure 3).
Figure 3. Auto-Catalytic Effect in the Linux Ecosystem
Figure 4 shows the history of LSB releases. LSB 4.0, introduced a number of major improvements:
Figure 4. LSB Release Timeline
Expanded coverage: seven completely new libraries and around 1,000 new interfaces were added as compared to its predecessor, LSB 3.2. Four completely new test suites and many new tests in the existing test suites expanded the test coverage significantly.
Version independence in tools: with the advent of backward-compatible standards starting with LSB 3.0, the proliferation of different LSB versions and the collection of corresponding tools and tests to support them, things were becoming confusing. LSB 4.0 introduced version-independent tools and tests that support all LSB versions starting from 3.0, which means users can select which LSB version they target when using the new tools, making it easy to move back and forth on the version stack as needed without the need for re-installation.
Best-effort dynamic linking: perhaps the largest difference between an LSB-compliant application and a native Linux application involves the use of the LSB dynamic linker. This has been used in the past to resolve difficult compatibility problems, allowing the native environment to differ from the LSB while still supporting LSB applications. LSB 4.0 debuted a new build strategy that allows an LSB application to run using the native environment when the LSB dynamic linker is not present.
Figure 5 shows the list of currently certified LSB 4.0 distributions.
Figure 5. LSB Certified Distributions
LSB version 4.1, released in March 2011, has the following main changes from LSB 4.0:
Additional symbols: a number of additional symbols have been added based on ISV demand. These are primarily symbols from glibc and related libraries, with a few new symbols brought in by uplifts of GTK and Cairo.
ALSA: ALSA has been a trial-use module in previous releases. For 4.1, the workgroup has promoted ALSA to a required component and included both shallow and normal-level tests for the ALSA interfaces.
Enhancements to printing: printing enhancements include the addition of ipp/HTTP interfaces from CUPS, as well as a GTK uplift that includes the common printing dialog.
The LSB workgroup has been working on the standard since the early 2000s, moving together with the evolving Linux ecosystem. We've learned a number of things along the way that are worth sharing with the community:
Need for upstream documentation: good standards should rely on a good documentation for the elements they define. In the case of LSB, the most burning point is poor upstream documentation of library interfaces. A number of good interfaces could not be included in the standard due to this reason. We would like to encourage upstream developers to pay attention to the documentation quality for their components as it helps develop consistent applications.
Test what you document: upstream components should have good functional tests that can check correctness of the component's implementation against the requirements stated in the documentation. Ideal conformance tests should have an explicit linkage between particular checks in the tests and particular requirements in the documentation, thus enabling traceability during testing.
New methods for developing interface standards: LSB is the biggest interface standard in the world. It embraces dozens of thousands of elements. Previous interface standardization efforts counted far fewer elements (for example, POSIX includes just about 1,800 interfaces). The huge number of elements makes it mandatory to use special methods and tools for developing the standard; otherwise, it is impossible to manage the complexity. The LSB workgroup, jointly with ISPRAS (Institute for System Programming of Russian Academy of Sciences), has developed a number of leading-edge methods and supporting tools to make development of the standard systematic and manageable. These include automatic generation of some parts of the LSB specification and tools, a systematic process and supporting tools for identifying candidates for inclusion into LSB and moving them through the standardization process stages.
Continuous testing—find problems early: our approach to testing has changed with time. In the past, our focus on testing has been limited to ad hoc QA on our own software and the auditing of results from customer tests. At some point, the LSB workgroup started running its own regular tests of both the enterprise and community distributions, as well as development versions of those distributions, and it has been filing bugs with distributions and upstream projects for found problems. These have resulted in quick deployment of fixes before they reach the public, both of our tests and of the products being tested.
The relationship between the LSB and the greater Linux community has been defined largely by the LSB's role as a trailing standard; we document and test for the behavior designed and developed by the community, rather than guiding the community to the behavior we design. On occasion, we act as a clearinghouse when different parties introduce incompatible behavior, but we intentionally avoid “kingmaker” roles even then, preferring to encourage the parties involved to come to a solution on their own.
Another of LSB's community benefits is our recently accelerated testing strategy, which has resulted in the discovery of numerous compatibility bugs at an early stage, allowing them to be fixed before they can make it into the wider world. To our knowledge, we are the only group performing wide multi-distro compatibility testing in the Linux community.
The challenge, going forward, is to communicate the value of the LSB to the Open Source Development community more clearly. To this end, we have been looking at making better use of social media, such as Facebook and Twitter, and in generating more statistics about our contributions.
A number of items on the rolling project plan page could be candidates for future versions of LSB. The workgroup generally finalizes the plan for the next release at a face-to-face session, weighing the demand/return for the development work to add new modules with the available resources we have to work on the LSB. Contributions to LSB are always welcome, and if a contributor was to provide a complete module with specifications and tests, it certainly would increase the chances of being added to the next version of LSB.
The LSB provides a standardized and tested common set of interfaces for ISVs to target, thus addressing the platform fragmentation problem. LSB's goal is to make the ISV's incremental cost of targeting each additional distro approach zero. The LSB Certification mark is available to distros and apps alike. By enabling ISVs to target the LSB rather than any one distro, the LSB enables portability across distributions. Also, the LSB serves as a place where distros, upstream maintainers and ISVs can come together to solve problems and prevent regressions. Interoperability often is more about having a forum to resolve differences rather than any one approach being correct.
LSB has gone through a long evolution together with the Linux ecosystem. Looking forward to advancing the standard further, the LSB workgroup appeals to the community to participate and contribute to the effort to help secure the success of the Linux platform as a whole.
The authors would like to acknowledge all companies, developers, community members, Linux Foundation members and employees, the ISPRAS team and everyone else who contributed to the LSB efforts. Such an industry-wide initiative as LSB would be impossible without the wide support and talented people involved.