Development and Testing cluster up

29 08 2007

I used to do Sakai cluster testing with multiple tomcat instances on the same box, and share the ports. This is Ok, but when 5 developers try to do the same thing on the same box you soon run out of meaningful port numbers, and a careless killall can bin a few hours worth of testing.

Hence the current interest in virtualisation in this blog. Well the cluster is now up and running, after getting rid of some problems with inconsistent arp responses. Each root partition contains java maven and tomcat and not much else. The source is built on a seperate server in a shared NFS mounted home directory and then with a few simple ssh  scripts the cluster nodes are updated automatically from that central location. This makes it much easier to restart the cluster and redeploy or reconfigure the cluster. Shared filesystem for content and search is provided by a NFS file system, and the MySQL instance runs on the same central box.
This is probably nothing like a production environment, but the aims are slightly different as the update cycle needs the same as for a single node developer environment. So far its working well, virtualisation is providing good separation between the cluster nodes, making good use of the resources and its easy to manage the root images on LVM with scripts. Recreating a new node now only takes a few minutes.

I havent tried yet, but I suspect the kernel and / filesystem required to bring java / tomcat up is very small.



2 responses

30 08 2007
Jason Shao

Can you really do cluster testing on 1 box? Even if it’s a multi-cpu box, aren’t most of your requests getting multiplexed into a single-stream anyway?

Having said that I’ve also been playing with virtualization in Dev & testing environments. Both VMWare (maybe eventually Xen, but we’re mostly a Solaris shop) and Amazon’s EC2 looks very nice.

31 08 2007
Ian Boston

Well your right about multiple tomcats on one box, but to a certain extend the packets will get interleaved so the request will come in slightly random order. On the Xen box I am using a Bridge configuration so the scheduling all happens at L2 in the stack, It would be the same as connecting the physical test boxes up to a Ethernet hub.

I am not really load testing here, which would be wrong. I am looking for deadlocks and strange behavior in a cluster.

I have used VMWare, and its fantastic…. but I wanted to see what Xen was like, partially because it was developed down the road at the Computer Labs, but also because its lower cost. I haven’t looked at EC2… but I will, thanks for the pointer.

%d bloggers like this: