Node.js vs SilkJS

28 09 2012

synchronous ducks

Node.js, everyone on the planet has heard about. Every developer at least. SilkJS is relatively new and creates an interesting server to compare Node.js against because it shares so much of the same code base. Both are based on the Google V8 Javascript engine that convert JS into compiled code before executing. Node.js as we all know uses a single thread that uses a OS level event queue to process events. What is often overlooked is that Node.js uses a single thread, and therefore a single core of the host machine. SilkJS is a threaded server using pthreads where each thread processes the request leaving it upto the OS to manage interleaving between threads while waiting for IO to complete. Node.js is often refereed to as Async and SilkJS is Sync. The advantages to both approaches that are the source of many flame wars. There is a good summary of the differences and reasons for each approach on the SilkJS website. In essence SilkJS claims to have a less complex programming model that does not require the developer to constantly think of everything in terms of events and callbacks in order to coerce a single thread into doing useful work whilst IO is happening. Although this approach hands the interleaving of IO over to the OS letting it decide when each pthread should be run. OS developers will argue that thats what an OS should be doing and certainly to get the most out of modern multicore hardware there is almost no way of getting away from the need to run multiple processes or threads to use all cores. There is some evidence in the benchmarks (horror, benchmarks, that’s a red rag to a bull!) from Node.js, SilkJS, Tomcat7, Jetty8, Tornado etc that using multiple threads or processes is a requirement for making use of all cores. So what is that evidence ?

Well, first read why not to trust benchmarks http://webtide.intalio.com/2010/06/lies-damned-lies-and-benchmarks-2/ once you’ve read that lets assume that everyone creating a benchmark is trying to show their software off best.

The Node.js 0.8.0 gives a request/second benchmark for a 1K response at 3585.62 request/second. http://blog.nodejs.org/2012/06/25/node-v0-8-0/

Over at Vert.x there was an of Vert.x and Node.js showing Vert.x running at 300,00 requests/s. You do have to take it with a pinch of salt after you have read another post http://webtide.intalio.com/2012/05/truth-in-benchmarking/ with some detailed analysis that points out testing performance on the same box with no network and no latency is theoretically interesting, but probably not informative for the real world. What is more important is can the server stand up reliably forever with no downtime and perform normal server side processing.

So the SilkJS benchmarks in one of its more reasonable benchmarks claim it runs at around 22,000 request per second delivering 13K of file from disk with a very high levels of concurrency 20000. Again its hard to tell how true the benchmark is since many of those requests are pipelined (no socket open overhead), but one thing is clear. With a server capable of handling that level of concurrency some of the passionate arguments supporting async servers running one thread per core are lost. Either way works.

There is a second side to the SilkJS claims that bears some weight. With 200 server threads, what happens when one dies or needs to do something that is not IO bound? Something mildly non trivial that might use a tiny bit of CPU. With 1 server thread we know what happens, the server queues everything up while the on server thread does that computation. With 200, the OS manages the time spent working on the 1 thread. There is a simple answer, offload anything that does and processing to a threaded environment, but then you might as well use an async proxy front end to achieve the same.

There is a second part of the SilkJS argument that holds some weight. What happens when 1 of the SilkJS workers dies? Errors that kill processes happen for all sorts of reasons, some of them nothing to do with the code in the thread. With 199 threads the server continues to respond, with 0 it does not. At this point everyone who is enjoying the single-threaded simplicity of an async server will, I am sure, be telling me their process is so robust it will never die. That may well be true, but process sometimes dont always die, sometimes they get killed. The counter argument is, what happens when all 199 threads are busy running something. The threaded server dies.

To be balanced, life in an async server can be wonderfully simple. There is absolutely no risk of thread contention since there is only ever one thread, and it doesn’t matter how long a request might be pending for IO for as all IO is theoretically non blocking. It doesn’t mater how many requests there are provided there is enough memory to represent the queue. Synchronous servers can’t do long requests required by WebSockets and CometD. Well they can, but the thread pool soon gets exhausted. The ugly truth is that async servers also have something that gets exhausted  Memory. Every operation in the event queue consumes valuable memory, and with many garbage collected system, garbage collection is significant. Although it may not be apparent at light loads, at heavy loads even if CPU and IO are not saturated, async servers suffer from memory exhaustion and or garbage collection trying to avoid memory exhaustion, which, may appear as CPU exhaustion. So life is not so simple, thread contention is replaced by memory contention which is arguably harder to address.

So what is the best server architecture for modern web application?

An architecture that uses threads for requests that can be processed and delivered in ms, consuming no memory and delegating responsibility for interleaving IO to the OS, the resident expert at that task. Coupled with an architecture that recognises long IO intensive requests as such and delegates them to async part of the server, and above all, an architecture on which a simple and straightforward framework can be built to allow developers to get on with the task of delivering applications at webscale, rather than wondering how to achieve webscale with high load reliability. I don’t have an answer, other than it could be built with Jetty, but I know one thing, the golden bullets on each side of this particular flame war are only part of the solution.

Advertisements

Actions

Information

2 responses

19 10 2012
Michael Schwartz

Thanks for the reasoned review.

When I wrote SIlkJS, I was not so much focused on the requests/second performance but concurrency. I was both surprised and pleased with the actual request/second results.

Initially, I ran ab tests against SilkJS to assure it could handle 2+ simultaneous requests. When I saw it do 2, then 10, then 50, then 5000 simultaneously, I just figured it was working pretty well.

When I saw that it was fast, I thought I’d compare against Apache. SilkJS is not as fast as Apache, but it’s pretty close (90%) for static content. For running business logic you’d do with Apache+PHP, SilkJS+V8/JavaScript is up to two orders of magnitude faster.

You mentioned many of SilkJS’s strengths. You wrote that SilkJS uses pthreads, which it doesn’t. It uses system processes via fork(), much the same way each tab in your browser is a separate system Process in Chrome. In a pthread application, a single thread can bring down the entire process (threads share the address space of the main program/process). SilkJS’s use of Processes allows any Process to die, get killed, crash, etc., and the other processes continue to run. Much like closing a tab in your browser, the other tabs continue to work.

A benefit of Processes you didn’t mention is garbage collection. The nature of V8 as a JavaScript engine is that it will choose to garbage collect freed variables and objects pretty much when it chooses. You can force a garbage collect to occur, which SilkJS does, and on a per Process basis. When one process is done serving all the keep-alive requests from a browser, it will garbage collect. Other Processes in other CPU cores will run just fine, the OS scheduler will allow a CPU core to part-time GC for one process and execute business logic for another, and so on. With a single threaded application, GC will kick in probably when you don’t want it to 🙂

As well, the SilkJS Processes will execute some predefined (configurable) number of requests from all sources/browsers and will then exit. The exit causes all memory used by V8 and your JavaScript code to be returned to the system. The other Processes continue to run, of course. The main program detects when a “child” process exits and spawns a new one to replace it. For these reasons, I believe memory usage for SilkJS may be more consistent and predictable.

I am a big fan of NodeJS. Where I think it excels is at massive numbers of long-lived connections like WebSockets or large file downloads. In production, though, I wouldn’t recommend either NodeJS or SilkJS or any other WWW server for static content. No matter how concurrent you can get NodeJS to be, a really big CDN is going to handle more connections and deliver content faster.

I’m a realist, though. Neither NodeJS nor SilkJS are going to cause major WWW sites like Yahoo! or Google or ESPN to junk their existing technology and rewrite it all in JavaScript. The trick is to figure out a place alongside that existing infrastructure to thrive, and I believe there’s room for both SilkJS and NodeJS.

Regards,
Michael Schwartz – Creator of SilkJS

19 10 2012
Ian

Michael,

Thank you for the clarifications and insights. Getting thoughts directly from the creator is always better than from any other source. Your comment on consistent and predictable memory usage is particular interesting, as I have noticed that although the event driven application environments don’t exhaust the available process or thread resources, they are particularly unpredictable when it comes to cpu and memory usage. Optimising apps to deal with that is hard, and I suspect there will be a whole new class of DDOS attacks that exploit that unpredictability to bring even the most capable event driven web servers to their knees. Any endpoint that performs mild computation or has any memory footprint is all that’s need.

SilkJS is great work, and it was refreshing to experiment with it.

Ian




%d bloggers like this: