Erlang vs node.js
I've written about Erlang in the past, and my suspicion that, soon or later, other languages/systems will come along and "eat its lunch". Scala is one such potential contender. Another that has been gaining some visiblity lately is node.js, a simple framework for creating networked applications on top of Google's V8 Javascript engine.
I should define what Erlang's lunch is a little bit better before we go on. Erlang does several things really, really well; better than many existing, mainstream systems:
- Concurrency - the Actor model that it uses is much easier and more pleasant than dealing with threads.
- Distributed systems - Erlang makes this fairly easy and pleasant as well.
- Fault tolerant systems - using its distributed powers, Erlang is good for writing things like telephone switches that can't spend any significant time down.
Of these, I think the big one is concurrency. Distributed systems are nice, but not a critical point for most people. Same goes with fault-tollerant: those who need it really need it, but the rest of us are willing to make some compromises. Our web applications are important, but generally not something where people's lives hang in the balance.
How does Erlang do "concurrency"? The developer creates Erlang "processes" that interact with one another by passing messages. These are not real, OS level processes, though, and this is critical to how Erlang operates. Since these processes must all coexist within one real, system level process, it is absolutely essential that no operation that they perform hangs the entire system! The Erlang runtime system is built around this concept. Any Erlang process can do things like read and write to the disk or network, or have a "while (true) { .... }" loop (it doesn't actually look quite like that in Erlang, but that's the idea), and it won't wedge the system. This knowledge is also critical when you want to interface Erlang to the outside world: if your C library contains a routine that might block for a long time, you can't just call it from Erlang as it won't be a well-behaved part of Erlang's world (there are ways around this of course, but make life a bit more complicated for the developer). All this is done with Erlang's scheduler: each Erlang process gets a number of operations it can run before some other process gets to run, so even our while loop will only run for a bit before the system moves on to something else. IO is rigorously done with non-blocking calls internally in order to keep that from becoming an issue.
No other system that I know of has such a focus on being non-blocking, and node.js is no exception: a while(true) loop is perfectly capable of wedging the system. Node.js works by passing functions (closures, in many cases) around so that work can be performed as needs be. However, the actual functions that run actually do block the system, and thus must be written in order to not run for too long. Also, and this is important, Node.js also does its best to make IO non-blocking by default, so that you don't have to worry about IO calls.
Node.js isn't up to the level Erlang is at, because it requires more manual intervention and thinking about how to do things, but it's probably "good enough" for many tasks. How often do you really write code with so many calculations that it slows things down? Not often in my case - the real world problem I most often encounter is IO, and node.js does its best to make that non-blocking, so that it can be handled in the "background" or a bit at a time, without wedging the system. And if you really needed to write a long-running calculation (say you want to stream the digits of PI or something), you can break up your calculation manually, which may not be quite as elegant as Erlang, but is "good enough" for many people.
"Good enough" concurrency, combined with a language that is at least an order of magnitude more popular than Erlang, and a fast runtime, combined with ease of use in general (it's way easier to get started with node.js than with most Erlang web stuff) make for a system that's likely to do fairly well in terms of diffusion and popularity, and is going to "eat some of Erlang's lunch". Or perhaps, rather than actually taking users away from Erlang, it's likely to attract people that might have otherwise gone to Erlang.
Trackbacks
Use the following link to trackback from your own site:
http://journal.dedasys.com/trackbacks?article_id=2186
-
This post was mentioned on Twitter by davidnwelton: Wrote a brief comparison of Erlang and node.js: http://bit.ly/cYLJHI
about {{count}} hours later:
I feel hungry now.
about {{count}} hours later:
I think you need to explain how node.js works for the web. I have never used it, but your description makes it sound like concurrency is handled on node.js' side while it is possible for an infinite loop to lock up the whole system.
From my point of view, that sounds like a terrible idea because it would mean a single hanging request could freeze the rest of the server.
Even Apache's approach to PHP is saner (spawn a lot of OS threads) in that perspective.
Is there something I don't understand?
about {{count}} hours later:
Highly interesting blog posting, thanks.
This sounds pretty much like the cooperative versus preemptive multitasking model in operating systems, with Erlang or node.js being the "operating system" juggling the various tasks.
In operating systems, preemptive multitasking pretty much prevailed, so that's an argument pro Erlang. But it could happen that a language like node.js comes along and, despite featuring concurrency which is inferior to Erlang, manages to spark a hype, greatly reducing the market share of Erlang. E.g., when MySQL started to become popular ten years ago, anyone experienced with databases cringed at its lack of foreign keys, transactions etc. which were only grafted onto MySQL later in an arguably half-assed fashion, but yet it became enormously popular, putting more serious RDBMS's like PostgreSQL in a fringe position. That is what could potentially happen to Erlang.
about {{count}} hours later:
don't make infinite loops.
please post a benchmark of mochiweb vs node.
about {{count}} hours later:
Most operations wait on I/O which will not block Node.js. Like any other server, handling CPU intensive work is best suited to worker processes, local or remote. Web Workers for Node.js, job queues like Resque and Beanstalk, and even AMQP would work fine. In short, it is a non-issue that Node.js is single threaded.
about {{count}} hours later:
@anon: What if it were waiting for a reply of any kind? Hanging for some resource? An exception causing a crash? This stuff can happen more frequently than infinite loops.
@Brian I'm not sure I understand. Are you saying node.js should do no work except sending messages on some kind of queue and waiting for something else to do the work?
And being single threaded is an issue. What if there's a fatal error in one of the parts served by node.js? Is it handled by the scheduler or something?
There's usually a reason why the multiprocess approach makes sense with PHP, Python or any other language. It's meant to keep errors from propagating from one request to the other. A single-threaded approach might work for a dispatcher, but I do not expect it to be solid when it has workers doing actual work.
about {{count}} hours later:
Personally, I'm not terribly worried about Node.js infringing too much on the space currently occupied by Erlang. However, the rate at which Erlang's share continues to grow will certainly depend on how well other languages manage to address concurrency issues.
The problem of having to ensure that no function may block the system is actually far worse than you'd think. I gave a talk on that exact topic at QCon 2010. http://qconlondon.com/london-2010/file?path=/qcon-london-2010/slides/UlfWiger_DeathByAccidentalComplexity.pdf
I don't know if the approach on slides 24-26 would be applicable to Node.js - if so, at least the worst kind of complexity explosion can be fought if sufficient care is taken.
The thing that attracts people to Erlang is that it has a consistent approach to building messaging systems, that neatly addresses aspects like robustness, complexity and scalability; and there is a knowledgeable and helpful community around it too. Node.js seems to address a different niche, so the two should be able to coexist quite nicely.
about {{count}} hours later:
As for non blocking IO, it can't be actually achieved without using OS-level threads. Why? Because IO can happen in a page fault handler and there is nothing that can be done with that except using another thread. Therefore I think language runtimes should not attempt to do their own scheduling, but rely on the OS.
It should be added, that doing IO in page faults (via mmap) is the most efficient way of doing it, because it saves some copying around (in Linux and most unices anyway -- Windows is a different story (especially on Windows CE mapping files sucks big time)).
Ok, that means relying on OS can be done only where the OS is actually up to the task. Unfortunately Windows often are not.
{{count}} days later:
Exposed Filters
These filters are displayed to the user visiting the page. For instance say you added all your node types to the filter above (i.e. page, mail, image, node) and set it to an exposed filter then you could let the user select the particular type from a pull down menu. I suggest learning the basics of views first before you start playing around with exposed filters.