Tuesday, April 07, 2009

Developing with OSGi and Apache Sling

These are notes I compiled while working on a project which uses OSGi (Apache Felix) and Apache Sling as the foundational framework. Sling is essentially Felix with JCR and templating bundles installed and a REST bundle. I will tell you how I got going with the basics of OSGi and Sling and some of the issues I ran into. First a few important links:
Before we really get started I want to mention something about OSGi in general. OSGi is basically a system for making Java code modular and handling isolation of dependencies (no slop allowed). This is done via a lot of really complex classloader graphing. A chunk of code in OSGi is called a module and a system will typically be made up of many modules. Inside modules there are packages which can be used internally (private), exported so others can use them (this is how services dependencies are shared), and imported (this is how a bundle would get the dependencies it needs to use an external service). OSGi completely manages the lifecycle of the the bundles in the system. Bundles define an activator (implements BundleActivator) which allows the developer to control startup and shutdown actions and register services. The modular nature of OSGi means services could go away at any time (or may not be started when your bundle is starting). Because of this, bundle code should be written to expect that a service might not be ready to use when it is starting. Waiting for services to be available is bad also because OSGi activators are expected to be quick so delaying them will cause failures. This generally means using the listeners and trackers provided by OSGi. The practice is called the Whiteboard Pattern (basically inverted listeners, those familiar with EntityBroker/EntityBus will recognize this as how providers work). These links are a good intro and you will want to be familiar with this conceptually before you attempt to create your first real bundle (helloworld not included):
http://www.theserverside.com/tt/articles/article.tss?l=WhiteboardForOSGi
http://www.knopflerfish.org/osgi_service_tutorial.html

Here are the steps I took to get Sling up and running (more details on the Sling site):
  1. Checkout the source code using subversion:
    svn co http://svn.apache.org/repos/asf/sling/trunk sling
    (I checked out revision 758703 since the current revision was not working)
  2. Build the source using maven 2:
    cd sling
    mvn clean install
  3. Run Sling using the built in Jetty webserver (as an executable jar):
    java -jar launchpad/app/target/org.apache.sling.launchpad.app-5-SNAPSHOT.jar
    (Note: the version of the jar will change over time)
    (you can also run drop a sling war into your own servlet container)
    (you can change the port by adding "-p #" (where # is the port number like 9090))
    (you can cause all logs to go to the console by adding "-f -")
    (you can change the logging level of sling by adding "-l #" (where # is 0-4 with 4=debug))
  4. Test out sling using your web browser:
    Go to http://localhost:8080/
    Click on the console link (login as admin/admin)
Now, if you are going to do anything in Sling (or Felix) at all you are likely to need to debug it, so here are the steps to use a debugger with Sling (generally applies to any java app really).
  1. Run Sling with debugging options enabled:
    java -Xdebug -Xnoagent -Xrunjdwp:transport=dt_socket,address=9001,server=y,suspend=n -jar launchpad/app/target/org.apache.sling.launchpad.app-5-SNAPSHOT.jar
    (You should see this in the logs: Listening for transport dt_socket at address: 9001)
  2. Attach the debugger from eclipse (or use whatever debugger you like):
    Run -> Debug Configurations
    Right click Remote Java Application -> New
    Connect tab: Set Port to 9001
    Source tab: Add the imported Sling code and your OSGi bundle project
    Click the Debug button at the bottom
    (You should not get an error if everything connects up)
  3. Place a breakpoint in Runtime class on the gc method
  4. Click on System Information and then the Run button next to Garbage Collection
    (the debugger should pick up the call and pause the JVM at the breakpoint)
  5. Use the debugger to trace through if you like (but bear in mind that the admin console is not actually part of Sling, it is part of Apache Felix so you will need the source for Felix)
Now you probably want to deploy a bundle so here are the steps to make a really simple one you can build and deploy into sling. I use the maven-bundle-plugin for this example, see the links for more details about it:
Here is the part of the maven pom that configures the bundle manifest:
<dependencies>
<!-- OSGi -->
<dependency>
<groupId>org.osgi</groupId>
<artifactId>osgi_R4_core</artifactId>
<version>1.0</version>
</dependency>
<dependency>
<groupId>org.osgi</groupId>
<artifactId>osgi_R4_compendium</artifactId>
<version>1.0</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.felix</groupId>
<artifactId>maven-bundle-plugin</artifactId>
<extensions>true</extensions>
<configuration>
<instructions>
<Bundle-Activator>org.azeckoski.osgi.SampleActivator</Bundle-Activator>
</instructions>
</configuration>
</plugin>
</plugins>
</build>
  1. Checkout the sample bundle code:
    svn co https://source.sakaiproject.org/contrib/caret/osgi-sample/tags/sample-1.1/ osgi-sample
  2. Build the bundle using maven 2:
    cd osgi-sample
    mvn clean install
  3. Access the Felix console for bundles (included in Sling):
    http://localhost:8080/system/console/bundles
  4. Browse and select the bundle (target/sample-1.1.jar)
  5. Click Install or Update and then Refresh Packages
  6. Scroll down to the Sample OSGi Bundle
    If it is listed then you have a properly installed bundle!
  7. Click the Start and then Stop buttons to the right of the bundle
  8. You should see something like this in the logs:
    Sample starting at: Mon Apr 13 13:15:27 BST 2009
    Sample stopping at: Mon Apr 13 13:15:29 BST 2009
  9. Now click Start to make sure the bundle is running
  10. Restart Sling (use the admin console or just kill it and rerun the command)
  11. You should see that the bundle is running (it was started automatically) and in the startup logs the Sample starting... should appear
Here is the (very) simple code from the activator class (comments removed). It just prints out a message when the bundle starts and another when it stops.
import java.util.Date;
import org.osgi.framework.BundleActivator;
import org.osgi.framework.BundleContext;

public class SampleActivator implements BundleActivator {
public void start(BundleContext context) throws Exception {
System.out.println("Sample starting at: " + new Date());
}

public void stop(BundleContext context) throws Exception {
System.out.println("Sample stopping at: " + new Date());
}
}
Congratulations. You have installed you first bundle. If you want to try out the debugging you can put breakpoints in the start and stop methods. It only took me a couple days to get to this point which I think is not great so I hope this tutorial has really sped up the process for you.

Now for the advanced bit. Creating your own bundle. I have a somewhat realistic set of bundles where one depends on another and one uses the http.service (and other bundles) so chances are you won't face things that are too much more complex than this.
There are a few things that I learned going through this that were not all that apparent so hopefully I can save you some digging with these pro-tips:
  • OSGi start order does not guarantee service start order - The order which bundles start does not guarantee that their services will also be started. In fact, relying on this is a mistake since OSGi is meant to be modular and services could go away at any time (see the note at the top about OSGi and whiteboard pattern). Code should be written to expect that a service might not be ready to use when the bundle starts. Waiting for services to be available is bad also because OSGi activators are expected to be quick so delaying them will cause failures.
  • Bundles should import the packages they export - This is not obvious but it makes sense when you think about it. Since there may be other bundles of higher rank that are exporting packages that match yours, you should use the highest priority, even in your own bundle. This behavior is actually taken care of my most bundle making tools automatically so just don't be surprised to see your packages in the imports.
  • javax packages will need to be imported - Only java.* is available to your bundle by default so all the dom,xml,net,wtc. stuff from javax has to imported. If you do not import it and happen to use it then your bundle will fail at runtime. Luckily, most OSGi installations have a core which already exports most of this stuff so just put it into your import-package as optional like so (just an example):
    org.w3c.dom;resolution:=optional,org.xml.*;resolution:=optional,javax.*;resolution:=optional,*
  • Services in the manifest are deprecated - As of OSGi R4, listing services (using export/import-services) is deprecated. All exports/imports are handled as packages now and the registration of the services is done in the activator.
  • Felix shell access inside Sling - There is a felix remote shell which can be used to access the felix shell service. It can be installed by just installing a couple bundles (shell, shell remote). This will allow you to run felix shell commands inside Sling via telnet.
I strongly suggest you use code that you know works already when building your first functional bundle. Pick something that does not have a lot of dependencies as well. For the following tutorial steps I will take you through the download, build, activate, and test process with my bundles and then look at some of the code.
  1. Checkout the sample bundle code:
    svn co https://source.sakaiproject.org/contrib/caret/osgi-eb/tags/eb-1.0/ osgi-eb
  2. Build the bundle using maven 2:
    cd osgi-eb
    mvn clean install
  3. Use the Felix console for bundles to install and start the eb-services and eb-rest bundles:
    (should be eb-component/target/eb-services-1.0.jar and eb-webapp/target/eb-rest-1.0.jar)
  4. Verify they started by refreshing and making sure they appear to be running:
    For some reason there are no errors logged to the console by default when a bundle fails to start and no messages appear in the web interface. As a result you simply get a silent failure where it seems to have worked but is actually just installed and inactive. The bundle should say Active next to it (make sure you refresh).
    NOTE: The errors will be logged into SLING_HOME/logs/error.log
  5. Go to the URL that is setup for the bundles (http://localhost:8080/eb), it should load up the description page for the rest webapp portion
For this sample code I have used the OSGi ServiceTracker and ServiceListener with a bit of utilities scaffolding (ServiceTracker2 and ServicesTracker) to setup my bundles and services in the whiteboard pattern. While this works fine for a smaller use case, I would suggest looking into OSGi DS (Declarative Services) and the Maven SCR Plugin when working with a large number of bundles and services.

Here is the start method from the EB services activator showing an example of how services are registered and tracked. This also demonstrates one way to track things which are used by our services. Pre-OSGi Java programming might require the EntityProvider to be manually registered with the EB system using a register method. By leveraging OSGi's ability to lookup services, we can instead simply have developers create their providers and register them with OSGi as services. Then the eb-services bundle picks up the registered providers and handles the registration automatically.
public void start(BundleContext context) throws Exception {
System.out.println("INFO: Starting EB module");

// Create the EB core services
coreServiceManager = new EntityBrokerCoreServiceManager();

// register trackers to handle the optional services
eipTracker = new ServiceTrackerPlus<ExternalIntegrationProvider>(context, ExternalIntegrationProvider.class) {
@Override
protected void serviceUpdate(ServiceEvent event, ExternalIntegrationProvider service)
throws Exception {
coreServiceManager.getEntityBrokerManager().setExternalIntegrationProvider(getService());
}
};

// register a tracker for the entity providers (will find them as osgi services and handle registration)
providerTracker = new ServiceTrackerPlus<EntityProvider>(context, EntityProvider.class) {
@Override
protected void serviceUpdate(ServiceEvent event, EntityProvider service)
throws Exception {
EntityProvider provider = getService(event);
if (event.getType() == ServiceEvent.UNREGISTERING) {
coreServiceManager.getEntityProviderManager().unregisterEntityProvider(provider);
} else {
coreServiceManager.getEntityProviderManager().registerEntityProvider(provider);
}
}
};

// look up a service we optionally want to use but do not require
ServiceTrackerPlus<DeveloperHelperService> dhsTracker =
new ServiceTrackerPlus<DeveloperHelperService>(context, DeveloperHelperService.class);
DeveloperHelperService dhs = dhsTracker.getService();
System.out.println("DeveloperHelperService is currently: " + dhs);

// register the core services from this bundle
ebRegistration = context.registerService( EntityBroker.class.getName(), coreServiceManager.getEntityBroker(), null);
ebManagerRegistration = context.registerService( EntityBrokerManager.class.getName(), coreServiceManager.getEntityBrokerManager(), null);
ebProviderManagerRegistration = context.registerService( EntityProviderManager.class.getName(), coreServiceManager.getEntityProviderManager(), null);
ebEVAPManagerRegistration = context.registerService( EntityViewAccessProviderManager.class.getName(), coreServiceManager.getEntityViewAccessProviderManager(), null);
ebHSAPManagerRegistration = context.registerService( HttpServletAccessProviderManager.class.getName(), coreServiceManager.getHttpServletAccessProviderManager(), null);

System.out.println("INFO: Started EB module and registered services");
}

In this code example, we see the code to handle the start of the EB rest activator. This allows the eb-services bundle to be stopped and started without having to manually do anything to the rest bundle (whose services depend on the ones in the eb-services bundle). It will shutdown the rest services until the core services it requires are available again.
public void start(BundleContext context) throws Exception {
System.out.println("INFO: Starting EB ReST module");
// initialize tracker
this.requiredServicesTracker = new ServicesTracker(context, HttpService.class, EntityBrokerManager.class) {
@Override
protected void requiredServicesReady(Object service, ServiceTracker2 changed,
Map<String, ServiceTracker2> serviceTrackers) throws Exception {
// required services are ready so startup
startServices(getService(HttpService.class), getService(EntityBrokerManager.class));
}
@Override
protected void requiredServicesChanged(Object service, ServiceTracker2 changed,
Map<String, ServiceTracker2> serviceTrackers) throws Exception {
// required services changed so stop and restart
stopServices(getService(HttpService.class));
startServices(getService(HttpService.class), getService(EntityBrokerManager.class));
}
@Override
protected void requiredServicesDropped(Object service, ServiceTracker2 changed,
Map<String, ServiceTracker2> serviceTrackers) throws Exception {
// required services gone so shutdown
stopServices(getService(HttpService.class));
}
};

// optional services trackers
dhsTracker = new ServiceTrackerPlus<DeveloperHelperService>(context, DeveloperHelperService.class) {
@Override
protected void serviceUpdate(ServiceEvent event, DeveloperHelperService service)
throws Exception {
DeveloperHelperService dhs = getService();
if (servlet != null) {
servlet.updateDHS(dhs);
}
}
};
// TODO need to handle the case of the hsapm changing?
hsapmTracker = new ServiceTrackerPlus<HttpServletAccessProviderManager>(context, HttpServletAccessProviderManager.class);

// register the servlet if services are ready
this.requiredServicesTracker.startCheck();
}

Here are some more links to OSGi materials that may be helpful:
http://neilbartlett.name/blog/osgi-articles/
http://www.osgi.org/About/HowOSGi

NOTE: Updated for the graduation of sling and changes in URLs that resulted

No comments: