JCR SQL Queries

5 02 2009

Care needs to be taken

SELECT *  FROM nt:base  WHERE jcr:path LIKE '/userenv/%'

works, but

SELECT *  FROM nt:base  WHERE jcr:path LIKE '/userenv%'

does not.

K2 Search

2 02 2009

Search is all inside Jackrabbit inside K2, there is no search service and no managing segments. It supports all standard documents that you might encounter, and the lag between update and search is generally < 100ms. This is not really any surprise  since the Query mechanism inside Jackrabbit depends on search. This means we can do relational queries using the search engine without hitting the DB at all…. just a bit more scalable.

K2 Memory Usage

2 02 2009

33MB Perm Space, 11MB Heap after startup, thats with most of the functionallity to support the UX project.

See attached from yourkit of the startup,


JPA EntityManagers

27 01 2009

Been having problems with JPA EntityManagers recently in K2 as we didn’t realise they should not be shared between threads. Fortunately fixing this in K2 doesn’t require very much. Create a proxy to a thread bound entity manager contained within a ThreadBound holder. When these are put into a request scope cache from the CacheManager, the ThreadBound.unbind method is called when the request ends, so we have a hook to perform commit and close down on the EntityManager. The really nice thing about using a Cache from the CacheManager is that it can also be created with a Thread scope rather than a Request scope. In Thread scope, its not unbound… so by changing one switch we can, if the JPA usage is well behaved move from a per request to per thread binding. Fortunately the overhead of creating an EntityManger from most JPA providers is < 1ms.. and with this approach, we don’t need to create it on every request.

Kernel Update

26 01 2009

That ssounds like a security patch, but its not :). The Sakai Kernel work is moving forwards. We have a demo server up and running with the UX work on top of it and should soon have Content Authoring functional. We have tested some of the aspects. Loaded 5000 users, a bit slow at the moment, but then we dont have any indexes in the DB. 0.4s per user. We have tested webdav, which looks just fine with 2G of files form 1K to 100K in side, and done some tests with large files > 1G. So far so good. The best bit appears to be it is quite happy in 64M of memory and starts in 12s.

This week I hope to get the site services sorted out (perhase today). We have had Carl Hall from GATech and Jon Gorono form UCD over in Cambridge which has helped immensely to make more progress, share thoughts and learn from each other.

There is more to come as this ramps up.

What not to do at an Airport ?

4 01 2009

Easy… arrive 5 hours after the plane takes off on New Years Eve… in Sydney… bound for London with no flights going out :). That’s what I did, oops. My family, who were with me were not that happy, but the consolation was we saw the fireworks on Sydney Harbour Bridge.

How did it happen ? An SMS was sent from the carrier (Singapore Air) telling me the flight time had changed, what it didn’t tell me was that this was the connecting flight from Singapore to London, and I didn’t check. Perhaps they should think through their UX a bit more and the impact of a small screen on the information they are trying to deliver. I will check next time.

What happened next? Well I was expecting a complete nightmare, but although this was entirely my fault (so I had no right to rant, and didn’t), and there were no seats available until 12 days later, a very nice bloke at the Sydney Singapore Air office (Davin he said his name was) got us all onto a flight 24 hours later, and we didn’t have to wait on standby at the airport. I guess some lucky person got bumped up to business to make space for us. 24 hours later, having done some work, watched a few movies and one embedded Linux reboot from the onboard video system (A380’s have 1 CPU with a full IP stack and Ethernet port per seat running a Linux kerne and every seat has 110v, if only I had a network cable) we arrived in London, tired, cold but very grateful to Singapore Air. The only shock was the temperature difference. 30C.

Apple Mail Sync Hang

30 12 2008

Sometimes when offline I have noticed Apple Mail hang, especially on slow connections with lots of latency. It appears that some ADSL providers in Australia route their US bound traffic via Japan, giving high packet latencies over the Pacific (400ms), so IMAP to GMail is slower than normal. If you have been working offline for some time, Apple Mail will store your operations in an offline cache in ~/Library/Mail/IMAP-XXXX/.OfflineCache where the   XXXX is the account name. In this folder are normally lots numbered files, and an operations file. The numbered files are emails messages, and the operations file is a redo log written in binary format referencing the numbered file. When Mail comes back online it will replay the operations file.

So, if you have a slow connection, and you see that an account (via the Activity Window) is blocking with some operation, then if everything else is OK (ie the connection) you can clear this operation by selectively removing the numbered file causing the block.

Obviously this is dangerous advice, as I can give no guarantees that you will be able to work out which file you should delete or if the operation that you are invalidating will be the right one, or what will happen next :), but it worked for me with an unintentional operation on a huge email with masses of attachments that would not allow the account to go online on a 6KB/s upstream link. So use with caution as a last resort when all else fails and you are prepared for all disasters 🙂

Adding a Branch to Git

21 11 2008

One of the benefits of Git is that its easier to manage and merge in multiple branches. You can do this with a svn git repo by pulling the svn branches into the git repo to perform merge operations locally, before committing back to the svn branch. Normally you might have pulled the whole tree from svn including the branches and tags with git svn clone http://myrepo.com/svn -T trunk -b branches -t tags but what do you do if you only took out the trunk and want to add the branch in.

In the git repo there is a file .git/config that contains a list of repote repos including the remote svn repos. git svn manages its operation by having remote branches in the git repo, you can see them with git branch -a but whatever you do dont touch any of the svn branches in your git repo.

So we have a git repo and we want to add a single branch in to be able to merge into that branch. Here is how.

1. Edit the .git/config file to add another remote repo

I am using the Apache Shindig repo, which I just did a branch on…. which is why I need to get it into my git repo.

        repositoryformatversion = 0
        filemode = true
        bare = false
        logallrefupdates = true
[svn-remote "svn"]
        url = https://svn.apache.org/repos/asf/incubator/shindig/trunk
        fetch = :refs/remotes/git-svn

I then add the new branch so my .git/config looks like this:

        repositoryformatversion = 0
        filemode = true
        bare = false
        logallrefupdates = true
[svn-remote "svn"]
        url = https://svn.apache.org/repos/asf/incubator/shindig/trunk
        fetch = :refs/remotes/git-svn
[svn-remote "svn081x"]
        url = https://svn.apache.org/repos/asf/incubator/shindig/branches/0.8.1-x
        fetch = :refs/remotes/git-svn-0.8.1-x

This adds the branch svn081x to my git repo, as a svn-remote and points it to the svn branch 0.8.1-x. It also gives it a fetch reference that is unique. At this point the git repo doesnt have anything in that branch so we need to fetch the branch, but we need to give git a bit of help at this point. We need to give it a revision on the branch so it can walk back down the branch to find the branch point. The revision of branch would also be Ok. svn log –stop-on-copy https://svn.apache.org/repos/asf/incubator/shindig/branches/0.8.1-x is going to help here  as this will list all the commits to svn on the branch.

2. Fetch the branch

git svn fetch svn081x -r719438

svn081x is the name of the remote branch in the .git/config and the revision is on the branch and close to the branch point. If you chose a revision that is not on the branch, you will get a url not found error from svn. If this works you will see the files in the branch being added to your local repo.

At this point you can see the branches you have with a

x43543:trunk ieb$ git branch -a
* master

Which shows the remote branches and the master which is active. Dont start working on the git-svn-0.8.1-x branch as its a remote branch that you should not touch. We need to create a git branch that is based on the remote first.

3. Create a branch from the remote branch

git checkout git-svn-0.8.1-x
git checkout -b master-0.8.1-x

The first switched to the git-svn-0.8.1.x branch and the second creates a new branch from the current branch called master-0.8.1-x. So we now have a working branch called master-0.8.1-x on which to work.

x43543:trunk ieb$ git branch
* master-0.8.1-x

git checkout master will switch the branch back to the trunk, and git checkout master-0.8.1-x will switch back to the branch. Now you can merge to your hearts content and git svn dcommit back to the svn repo when you are happy with the results.

Unit Test and Constructor IoC

28 10 2008

I might have said it before but, Constructor IoC has 2 big advantages over setter and getters.

  1. The IoC all happens in one shot, so there are no issues about a class being half configured, its always thread safe from the start.
  2. When doing writing Unit tests is really clear what is needed to make a class function
  3. Less code.

That was 3… oops… bloat 🙂

Increasing Code Coverage

25 10 2008

One of the problems for developers, is that the honest ones want to increase their code coverage and unit tests, but dont really know how much of their code isnt tested. There are solutions to this. You can add a coverage report to maven and build a site, or you can use an eclipse plugin, which is more interactive. http://www.eclemma.org/ works well inside eclipse and gives good information about the lines being covered. The temptation with this evidence that code is covered or not, is to chase the % coverage up, by devising unreal ways of exercising the code. This should be resisted at all costs, as it will give a false sense of security, but it would be get to the point where the last 5% of coverage is all there is to worry about.