Building OSGi bundles for the JDisc Container

The JDisc container uses OSGi to provide a modular platform for developing applications that can be composed of many reusable components. The user can deploy, upgrade and remove these components at runtime.

What is OSGi?

OSGi is a framework for modular development of Java applications, where a set of resources called bundles can be installed. OSGi allows the developer to control which resources (Java packages) in a bundle that should be available to other bundles. Hence, you can explicitly declare a bundle's public API, and also ensure that internal implementation details remain hidden.

Unless you're already familiar with OSGi, we recommend to see Richard S. Hall's presentation Learning to ignore OSGi. This presentation explains a lot about how OSGi works, and why. There are also other great OSGi guides available online.

For those familiar with OSGi, JDisc uses OSGi's module and lifecycle layers, and does not provide any functionality from the service layer.

What is an OSGi bundle?

An OSGi bundle is a regular JAR file with a MANIFEST.MF file that describes its content, what the bundle requires (imports) from other bundles, and what it provides (exports) to other bundles. Below is an example of a typical bundle manifest with the most important headers:

Bundle-SymbolicName: com.yahoo.helloworld
Bundle-Description: A Hello World bundle
Bundle-Version: 1.0.0
Export-Package: com.yahoo.helloworld;version="1.0.0"
Import-Package: org.osgi.framework;version="1.3.0"

The meaning of the headers in this bundle manifest is as follows:

  • Bundle-SymbolicName - The unique identifier of the bundle.
  • Bundle-Description - A human readable description of the bundle's functionality.
  • Bundle-Version - Designates a version number to the bundle.
  • Export-Package - Expresses which Java packages contained in a bundle will be made available to the outside world.
  • Import-Package - Indicates which Java packages will be required from the outside world to fulfill the dependencies needed in a bundle.

Note that OSGi has a strict definition of version numbers that need to be followed for bundles to work correctly. See the OSGi javadoc for details. As a general advice, never use more than three numbers in the version (major, minor, micro).

Building an OSGi bundle

As long as the project was created by following steps in the developing applications guide, the code is already being packaged into an OSGi bundle by the Maven bundle plugin. However, if migrating an existing Maven project, change the packaging statement to:

<packaging>container-plugin</packaging>

and add the plugin to the build instructions:

<plugin>
    <groupId>com.yahoo.vespa</groupId>
    <artifactId>bundle-plugin</artifactId>
    <version>current-version</version>
    <extensions>true</extensions>
</plugin>

Because OSGi introduces a different runtime environment from what Maven provides when running unit tests, one will not observe any loading and linking errors until trying to deploy the application onto a running JDisc Container. Errors triggered at this stage will be the likes of ClassNotFoundException and NoClassDefFoundError. To debug these types of errors, you need to inspect the stack traces in the error log, and refer to the OSGi troubleshooting document

vespa-logfmt with its --nldequote option is useful when reading logs.

The test suite needs to cover deployment of the application bundle to ensure that its dynamic loading and linking issues are covered.

Depending on non-OSGi ready libraries

Unfortunately, many popular Java libraries have yet to be bundled with the appropriate manifest that makes them OSGi-compatible. The simplest solution to this is to set the scope of the problematic dependency to compile in your pom.xml file. This will cause the bundle plugin to package the whole library into your bundle's JAR file. Until the offending library becomes available as an OSGi bundle, it means that your bundle will be bigger (in number of bytes), and that classes of that library can not be shared across application bundles.

The practical implication of this feature is that the bundle plugin copies the compile-scoped dependency, and its transitive dependencies, into the final JAR file, and adds a Bundle-ClassPath instruction to its manifest that references those dependencies.

Although this approach works for most non-OSGi libraries, it only works for libraries where the jar file is self contained. If, on the other hand, the library depends on other installed files, it must be treated as if it was a JNI library.

Depending on JNI Libraries

This section details alternatives for using native code in the container.

OSGi bundles containing native code

OSGi jars may contain .so files, which can be loaded in the standard way from Java code in the bundle. Note that since only one instance of an .so can be loaded at any time, it is not possible to hot swap a jar containing .so files - when such jars are changed the new configuration will not take effect until the container is restarted. Therefore it is often a good idea to package an .so file and its Java API into a separate bundle from the rest of your code to avoid having to restart the container on all code changes.

Add JNI code to the global classpath

When the JNI dependency cannot be packaged in a bundle, and you run on an environment where you can install files locally on the container nodes, you can add the dependency to the container's classpath and explicitly export the packages to make them visible to OSGi bundles.

Add the following configuration in the top level services element in services.xml:

		
<services version="1.0">		
    <config name="search.config.qr-start">		
        <container>		
            <classpath_extra>/lib/jars/foo.jar:/path/bar.jar</classpath_extra>		
            <export_packages>com.foo,com.bar</export_packages>		
        </container>		
    </config>		
    ...		
</services>		
Adding the config at the top level ensures that it's applied to all jdisc clusters.

The packages are now available and visible, but they must still be imported by the application bundle that uses the library. Here is how to configure the bundle plugin to enforce an import of the packages to the bundle:

		
<plugin>		
  <groupId>com.yahoo.vespa</groupId>		
  <artifactId>bundle-plugin</artifactId>		
  <extensions>true</extensions>		
  <configuration>		
    <importPackage>com.foo,com.bar</importPackage>		
  </configuration>		
</plugin>		
When adding a library to the classpath it becomes globally visible, and exempt from the package visibility management of OSGi. If another bundle contains the same library, there will be class loading issues.