Following a bunch of forks.

22 04 2009

Can start to be a pain, this fetches things a bit faster for review.

x43543-2:osgikernel ieb$ git branch -a
* master
  carl/master
  codders/master
  danatcaret/master
  daniel/master
  holdorph/master
  jholtzman/master
  origin/master
  stuartf/integration
  stuartf/master
x43543-2:osgikernel ieb$ for i in `git branch -a | grep '/master' | grep -v origin | cut -f1 -d'/'`; do git fetch $i; done
x43543-2:osgikernel ieb$ gitk -all

Which looks like this:

Tracking branches in GitK

Tracking branches in GitK

Advertisements




Bundle Dependencies

15 04 2009

Having spent a lot of time resolving bundle dependencies over the last two weeks here are some of the thing that I have learnt.

1. Use the BND tool and the maven bundle plugin to build the maifest, doing it by hand except for the simplest bundle is hard.

2. Dont assume that every  package needs to be imported. Things like commons-logging have 10’s of optional packages that you are probably never going to use. To filter these you can remove them with (maven-bundle-plugin)

<Import-Package>
!org.apache.avalon.*,
*
</Import-Package>

But remember to put the excludes (!) before the global include (*).

3. You never know how an OSGi container might be configured so don’t assume that everything jre comes from the boot classloader, it may come from a Framework bundle. You can use the resolution:=optional statement to ensure this is handled. Remember if its not imported into a bundle, then it wont be loaded.

<Import-Package>
!org.apache.avalon.*,
*,
org.xml.sax;resolution:=optional
</Import-Package>

4. If you are feeling really lazy and want the bundle to load but generate ClassNotFoundExceptions later then just use *;resolution:=optional on all imports.

5. NoClassDefinitionError means that something the class depends on was not imported into the package, but it is exported by annother bundle. Bundles are private parties, “If your not on the list your not comming in!” import == the list.

6. Package locally. If you have a jar that is private to the bundle and unlikely to be of use to anyone else, package it locally inside the bundle with <Embed-Dependency>, just because you can create a bundle, doesn’t mean you should.

7. Don’t assume that the OSGi framework is telling you the truth. Often a bundle will report an error like the NoClassDefinitionError. That bundle may not be at fault, look carefully at the stack trace to determine the class where the error happened, and then think carefully which classloader and hence bundle was in use when the error happened.

8. Dont assume that if you are having classloader problems or resolution problems that you can reload bundles. I have found the reloading bundles only works reliably when there are no classloader or resolution problems. When there are, its anyones guess what state the OSGi container is left in. I have been sutting down, and completely deleiting all state as the only reliable way of debuging dependencies, YMMV…. enjoy:)





bundle and dependency plugin

10 04 2009

I have been experimenting with getting a happy medium in the bundle and dependency plugin with respect to internal dependencies. The bundle plugin analyses the code base and constructs a suitable MANIFEST.MF for the bundle, and the dependency plugin embeds jars into the jar to resolve internal dependencies. Its handling of those jars is interlinked, and confusing when it comes to both dependencies and transitive dependencies. The dependency plugin may pickup your dependencies and copy them into the jar, but unless they are required by the directives in the bundle plugin they wont appear in the final bundle. So here are some settings that appear to work.

For the bundle plugin:

    <Embed-Dependency>*;scope=runtime|compile</Embed-Dependency>
          <Embed-Transitive>true</Embed-Transitive>

causes all dependencies marked as runtime or complie, and the transitive dependencies from those to be embeded into the bundle, if they were copied there in the first place.

In the dependency plugin:

<includeScope>runtime</includeScope>

Will cause all runtime and compile scope dependencies to be included in the packaging process to be later filtered by the bundle plugin.

Then elsewhere in the pom dependencies can be marked as compile or runtime appropriately.

You could use explicit definition with includeArtifactIds in the dependency plugin and the same explicit artifactId’s in Embed-Dependency statement in the bundle plugin, but that rapidly becomes error prone.





Be careful what you publish

10 04 2009

Google are now analysing videos uploaded for Copyright infringement. If you happen to video cam recording of anything that has some ambient music, and publish that, even to one person, I am reliably informed that, if the music was copyrighted, by publishing you have breached the copyright. Do it too many times and Google will ban you.

I video cam’d my daughters school play that had some background music, from the Jungle Book, the video was banned since Disney owns the copyright. I breached the copyright by publishing to her grand-parents even though I made the video private on Google Video. I have since deleted the video from Google. The school is probably in breach of copyright by pressing play without explicit permission form Disney Corp when parents were listening.

I have a feeling the days are numbered when Google can take a relaxed attitude copyright infringement on YouTube and Google Video, will this be the start of a contraction of services?





Loading from OSGi Framework bundles

10 04 2009

There are some really confusing things about the Class resolution in OSGi, that to the uninitiated like myself of 4 days ago appear like complete black magic. First off, there are 9 rules to OSGi class resolution to confuse you, if you were not already confused enough by classloading, but since I have struggled for 4 days, I thought I might share one solution.

java.lang.NoClassDefFoundError: org.xml.sax.SAXException

org.xml.sax.SAXException is exported from the Framework bundle (normally), so your bundle should be able to find it… actually it can, the NoClassDefFoundError unlike the ClassNotFoundException means it has been found but it cant be loaded. Why ever not ?

Normally this is because a component of the class in question is not available, like a static property for instance. SAXException turns out to be a good example, it has no static properties other than one long, and there is nothing outside java.lang. So there is no reason why it should generate a NoClassDefFoundError when its not generating a ClassNotFoundException.

It turns out the OSGi class search policy is blocking loading it. For any class from any package other java.* to be loaded from another bundle is *must* be imported. Although re-reading the 9 policies in the spec, I can see anything jumping out at me that states that.

Normally the maven bundle plugin would analyse all your classes and add an import, but if a bundled library contains a reference to a system class this will not be included in the import. So, and this is my theory, the class can be found (hence no ClassNotFoundException) but cant be loaded because its ‘helpfully’ (ho,ho,ho) blocked by the OSGi classloader search policy.

To fix this you must add explicit imports for the unknown classes that your third party libraries will load. eg

 <Import-Package>sun.io;resolution:=optional,
            javax.xml.parsers,
            org.w3c.dom,
            org.xml.sax,
            org.xml.sax.ext,
            org.xml.sax.helpers,
            *
            </Import-Package>

And then the bundle will have access to the classes exported from the Framework classloader.

All of what I have said above could be completely wrong, my next problem is org.w3c.dom.xpath not in the Framework bundle, but in xml-apis 1.3.04, incidentally along with the rest of the xml apis that I just imported from the framework.

To me this all sounds extremely dangerous, taking some classes from one jar and some from the JVM that are also in the jar, but then I am a novice.  I will post more as I discover it, partially because Google didn’t help me find a solution to this.

Update:

xml-apis is now bundles as a jaxp-api bundle all on its own so it provides the Non Framework parts of org.w3c and org.xml.

I have also found it necessary to bundle jsr-173 (javax.xml.stream) from xmlbeans as a seperate bundle.

But I am still getting the same NoClassDefFoundError errors, now traced back to the EclipseLink provided bundle containing the PersistenceProvider. You have to work out which bundle the class that caused the error came from, not the reported bundle in the stack trace, barking up wrong tree, yet again.

Update2:

You have to be really carefull, the NoClassDefError hapens when the class is being loaded by the classloader. If it depends on Framework classes, they must be listed in the import otherwise they will not get loaded. So you have to find the class thats failing to be instansiated, and then work out which bundle it came from before adding it to the list of imports. The BND tool should do this, but it cant dig into every possibility in a 3rd party library so it wont. Its quite a long winded process to get it right.





OSGi Service Model is limited

2 04 2009

I don’t want to sound negative, but as always when you look past the hype and read the detail some of the truth comes out. Take OSGi Configuration Service. On the face of it this would allow each service to have its own configuration and allow you to bring a service up and let is manage everything. Well to an extent that is true, provided you adopt the ManagedService model. This means that OSGi manages all the services for you, you define what you want to be configured by using constants within your code or creating an xml file defining he configuration constants, and OSGi takes care of the rest. Sounds pleasant enough, and allows changes in configuration to be listened to by the components. Update the config, and its reflected in the component. So this works perfectly well for a single component with a single service impl exposing a single api, but as soon as your bundle contains a collection of services implementations that are constructed with IoC, then none of the services can be managed. In short the ManagedService places a boundary around the service implementation that prevents it from communicating with other services except via static instances, or something horrible.

The impression that you could use the OSGi Configuration Service at run-time or in a cluster is somewhat mistaken. So I have 5 OSGi JVM’s in my cluster, and I change the config on one of them, magically the ManagedServices notice the change and reconfigure, but what about the other nodes in the cluster that now have a different configuration ?

And then there is the use case where we have configuration information that really shouldn’t be changed when a bundle is active. A database connection. Change that mid stream, who knows what will happen.

At the moment, unless anyone reading this can tell me how to use the configuration service where bundles expose multiple services, constructed by IoC, I feel its going to be simpler to create a ManagedService that is a configuration service, and allow that service to communicate throughout the cluster and manage reloading. Its a pity since I had hoped OSGi would manage this for me, perhaps I should have not trusted the hype again, reminds me of a belief in EJB many years ago.