GSoC Review

30 08 2011

This is the third or fourth year I have done a GSoC project as a mentor. They are always great fun, and just as hiring well reaps rewards, so does evaluating a GSoC student. Every year I have chose one with attitude over ability. The right attitude that is. An attitude to show those who appear to know more about the subject the evidence that proves they need to think again, and a hunger for the information that might add to what they already know. My wife always says men are strange. When they are lost in a big city they rarely ask the way. More often they search for a map or pull out a iphone or htc and try and work it out for themselves. All around them are experts that will not only tell them the way, but tell them about the great cafe to stop for coffee at one street up, and the dark alley to avoid one street down. Ability can be acquired, attitude is who you are. If its wrong, then it takes a long time to change. This year the GSoC student I chose and was mentoring was Aadish Kotwal. He knows how to ask questions and enrich his journey, in the process he enriched mine and showed me a thing or two. I think he enjoyed the process, and wrote some great code. I certainly enjoyed it. This is what he said:


According to the link [1] I have submitted my share of code in OAE project on [2]. Please let me know if you expect anything else.

Also, Ian I am so much grateful to you, you have no idea! Thanks a lot for passing me in final evaluations 🙂 It really wouldn’t have been possible without your continuous support and confidence in me. Open source world, though awesome, was really a bit intimidating. But it was because of your gentle attitude and kind understanding that I was able to come this way! I have learnt SO many thing from you and I am so grateful! I can just go on, but I know each passing minute costs you a lot!

But lastly and again a big thanks! I will try my best to fulfill yours and Amyas’s wishes. It has been an awesome experience working with you! I will try my best to stay in touch with you and try to contribute as much as I can if you wish!

[1] :

[2] : 
Aadish Kotwal”

 He deserved to pass with flying colors. Well done Aadish.

SparseMapContent 1.0 released

26 08 2011

Normally I would announce this sort of thing on a mailing list, but I dont have one, so this is the next best thing. SparseMapContent v1.0 has just been released. Its a project I started almost 12 months ago to enable Sakai OAE to store user generated content in shallow wide hierarchies, where 1000s of users would be performing updates, potentially served by an elastic cluster. It stores its content in a simple key value store with rdbms storage on Derby, MySQL, Oracle, PostgreSQL or column storage on Apache Cassandra or Apache HBase. In addition there are both bitstream stores for shared filesystem or as blocks within the the underlying column database. It comes bundled as an OSGi bundle intended to work with Apache Sling.

Released with SpraseMapContent is a Solr bundle based on a Snapshot of Solr 4, intended to provide a free text and keyword index for the content stored in SparseMapContent. That bundle maintains a persistent queue of pointers to content to be indexed filled by OSGi events, and indexer framework that resolves the pointers and indexes the content including metadata, and a search index. This bundle is also used by Sakai OAE, although like the SparseMapContent bundle it could be used by any application wanting to maintain a Solr index from an event stream.

If you dont like the names, neither do I and suggestions are welcome. Thank you to all those who have made contributions over the past 12 months.


Maven Repo:


Code bases, issue trackers, etc
License: Apache 2

A View of Human Society

18 08 2011

“Although it may be tempting to think of social institutions as functioning like organs, human societies behave, in practice, much more like slime molds. They don’t have eyes or brains that are anything like human eyes and brains. So although any one of us can learn from our mistakes, foresee problems and act reasonably to solve them, collectively we don’t do a very good job of this.

Granted, our species is very intelligent and has developed exquisite communication across the world and through the centuries. But while people are smart enough to anticipate problems, they are also smart enough to make counterarguments. Every good idea in history has had to fight against many bad ideas before winning broad acceptance.”

– Nathan Myhrvold 2011

Cambridge OAE: IMS-CP Import Support

15 08 2011

We have been working hard on IMS-CP at Cambridge. I should qualify that, Yushan Li from Tsinghua University who has been at Caret for the past month has been working hard on implementing IMS-CP import, and I have been causing trouble for him. Still, we have a working implementation of IMS-CP import that is in our “production” branch. It takes an IMS-CP file and during upload, unpacks the Zip file and converts the IMS Manifest into a Sakai Doc structure definition, so that the IMS-CP appears as a Multilevel Sakai Doc within the user library. They can still get to the original Zip file if they want to download it, and they can also use the Sakai Doc anywhere they can use any other Sakai Doc created inside OAE. Resources within the IMS-CP are also unpacked so that relative URLS within pages of the IMS-CP continue to work correctly.

To test this we have been using sample files created by Charles Sturt University from the Sakai CLE Melete Tool. Yushan’s graduate studies have including contributing to the new Metadata for Learning Resources standard and working closely with IEEE LOM, so there is now support for IEEE LOM within OAE at Cambridge and we handle some of the cases where the Melete tool exports diverge from the IMS-CP standard.

The following Screen shots show the functionality in action, uploading a content package which is converted on the fly into a Sakai Doc and then re-using that Sakai Doc in a Group. The main modification to the Sakai OAE code base is to deep levels of nesting within a Sakai Doc. Previously Sakai Docs could not nest more than one level deep. We hope to be adding IMS-CP export shortly, and then moving on to IMS-CC Import and Export. All of these features have been on the Nakamura Roadmap for almost two years now so is nice to see them being implemented, and we are very fortunate to have someone as knowledgable as Yushan doing this work.


Select Add Content

Locate and upload an IMS-CP package, this one was produced by Sakai CLE Melete/Modues Tool

The IMS-CP file appears in the Library as a Sakai Doc ready for use.

Viewing the IMS-CP Sakai Doc directly as a Content Pool item in the users Library

Embedding the IMS-CP Sakai Doc into a Group as a Page

The IMS-CP Sakai Doc embedded as a Page within a Group, notice the navigational structure of the IMS-CP is maintained

Cambridge OAE: Engineering Syllabus Widget

15 08 2011

For the deployment of OAE at Cambridge we needed and easy way which institutions could embed syllabus information into the Courses and Group pages. The Uniersity of Cambridge has many departments, each of which operates largely autonomously when it comes to the provision of teaching and learning information to its students. The Engineering department, which will be one of the early adopters maintains this information as web pages with a reasonably well defined structure. Other departments may have a simular approach. This widget enables a user to embed content from the Engineering Teaching pages directly into any page within the Sakai OAE instance at Cambridge, simply by selecting the year and lecture from a set of drop downs. The functionality is implemented as a widget using a template configured proxy on the back end. We use Google’s Javascript Caja implementation to sanitize the HTML, part of the Sakai Widget API and we parse the html to remove headers and footers. Development for this widget took about 6 hours to complete. No back end functionality was required.

The following sequence of screenshots shows a Sakai Doc being created in a Group, then the inline content widget being added and configured to inline the Lecture information in question.

Insert the Inline Content (Engineering Syllabus) widget into a Sakai Doc added to a Group

Select the year and paper to be displayed, from a set of configured papers.

Verify that the paper is correct from the preview that is presented.

The Configured widget appears as an Icon in edit mode.


Saved page contains the Page from the Year 1 Syllabus fetched from the Engineering Department Teaching pages.

Cambridge OAE: Accept Terms and Conditions Widget.

15 08 2011

For the University of Cambridge OAE instance we have an “Accept Terms and Conditions Widget”. This is loaded as a widget on all pages and checks that the user has accepted the current terms and conditions. I they have not, it pulls their official identity from the institutional LDAP, and asks them a) do you accept the terms and conditions and b) are these the details that should be used. The wording of the terms and conditions is being reviewed by the lawyers at the moment, and can be internationalized as required. Development took about 4 hours, start to end and it uses most of the Sakai Widgets API. There is one small patch to the core code base to ensure that a div appears on every page to load the widget. Code is in the Cambridge OAE extension repositories , and is deployed as separate jars into  instance.