I knew there was something wrong. You know that gut feeling you have about something, when it just doesn’t feel right, but you cant explain coherently enough to persuade others so eventually self doubt creeps in, and you go along with the crowd. We have a mantra, borrowed phrase really, borrowed from JCR. “Content is everything”. Its possible it was borrowed without knowing what it really meant. One of those emperors new clothes things, this time the emperor really was wearing cloths, and they were so fantastic that perhaps we thought they would fit our build, so we borrowed them, not quite realising what that meant.

One of the founding principals of Sakai 3 is that content should be shared everywhere. That expands to being re-used, repurposed, reowned everywhere. To achieve that, there are two solutions give content ownership and allow users who own the content to organise and manage that content how they feel fit, including adding their own structure to the content if they feel thats what the want to do. Alternatively, put all the content into a big pool and let users find it, tag it, apply ontologies to it and crucially express access rights at the individual item level. Neither approach is right, neither is wrong. The former has compromises when ownership is changed the latter has compromises when each uses needs to manage large volumes of content individually. I am not going to say which is better, been burnt too many times trying to apply engineering logic to user experience design decisions, but one thing I do know is that the underlying implementation for one approach is very different from the underlying implementation for the other. Getting them the wrong way round is likely to lead to problems.

So the UX design process decided that the big pool approach was right. Often quoted in discussions was doing things like Google Docs. Handing round pointers to documents, identified by a key at the end of each URL opaque in meaning to the end user, but immensely scalable. Expressing access rights as “share this item with my friends”, “make this public”, or “I’m happy for anyone who has this URL to edit this”. In that there was no expression of where the document lived, no organisation of the document into containers and certainly no management of access on containers, although it interesting to see that the limitation of managing large volumes of documents in a flat list has lead Google Docs to introduce folders where like minded documents get shared with collaborators by virtue of their location. The content model is scalable on two levels. On a technical level, its easy to generate billions of non conflicting keys per second and its easy to shard and replicate the content associated with those keys on a global scale migrating information to where its needed avoiding the finite limitations of the speed of light and routers. On a human level the machine generated key and per user hierarchy eliminates all conflicts. Google would be bust if it offered help desk resolution of conflicting document URLs where 2 users demanded to share the same web address for their document. By allocating gmail username on a first come first serve basis, Google managed to get tacit acceptance of a given naming scheme, without helpdesk load. How do you persuade Google to give you the userid of another user, just because you believe you have a right to it, and they got there first? Dream on ?

We chose a content technology because we wanted something that worked really well and covered the generalised use case of Content Management. The content system we have is hierarchical and maps URLs directly to content paths, exploiting hierarchal structure to make it easier to manage large volumes of data. It hits the sweet spot of Enterprise Content Management, but it it right for us ? In Sakai 3, UX design has chosen a pooled content model for all the same use cases as covered by Google Docs, but we are building it on a content system that requires agreement of URL space, agreement of locations within the content system and critically uses that hierarchy to drive efficiencies on many levels. Hierarchy is fundamental to the Java Content Repository specification, fundamental to Jackrabbits implementation and to some extents a natural part of the http specification. Attempting to layer a pooled content system over a fundamentally hierarchical storage system is probably a recipe for disaster on two levels. Technically it can be done, we have done it, but as my gut tells me we are beginning to find out, it wasn’t a good idea. All those efficiencies that were core to the content repository model are gone, exposing some potentially quite expensive consequences. At a human level we have side stepped the arguments over who owns what url in a collaborative environment by obfuscating the URL, but in the process we have snatched back from the user the ability to organise their content by hierarchy. The help desk operations that support Sakai 3 wont go bust processing URL conflict resolution since users dont get URLs to be conflicted over.

What should we do? I think we should admit that the models are separate and not try and abuse one user experience with the wrong supporting implementation or conversely force an implementation to do what it was not designed for. We have a crossover and we have to choose. For want of better words, we have to chose Content Management User Experience supported by content management storage living true to everything is content embodied by David’s model, or Pooled User Experience supported by UUID based object storage intended for massive scale. Mixing the two is not an option.

Hindsight is a wonderful thing to learn from; mistakes made ? Yes, we have mixed up a choice on a technical level with political desires forgetting that in a design led development process, technical decisions must be made purely to service the design. If the political desires were important, they should have adjusted the design process from the start. I live and learn.