Erlang vs node.js

I've written about Erlang in the past, and my suspicion that, soon or later, other languages/systems will come along and "eat its lunch".  Scala is one such potential contender.  Another that has been gaining some visiblity lately is node.js, a simple framework for creating networked applications on top of Google's V8 Javascript engine.

I should define what Erlang's lunch is a little bit better before we go on.  Erlang does several things really, really well; better than many existing, mainstream systems:

  1. Concurrency – the Actor model that it uses is much easier and more pleasant than dealing with threads.
  2. Distributed systems – Erlang makes this fairly easy and pleasant as well.
  3. Fault tolerant systems – using its distributed powers, Erlang is good for writing things like telephone switches that can't spend any significant time down.

Of these, I think the big one is concurrency.  Distributed systems are nice, but not a critical point for most people.  Same goes with fault-tollerant: those who need it really need it, but the rest of us are willing to make some compromises.  Our web applications are important, but generally not something where people's lives hang in the balance.

How does Erlang do "concurrency"?  The developer creates Erlang "processes" that interact with one another by passing messages.  These are not real, OS level processes, though, and this is critical to how Erlang operates.  Since these processes must all coexist within one real, system level process, it is absolutely essential that no operation that they perform hangs the entire system!  The Erlang runtime system is built around this concept.  Any Erlang process can do things like read and write to the disk or network, or have a "while (true) { …. }" loop (it doesn't actually look quite like that in Erlang, but that's the idea), and it won't wedge the system.  This knowledge is also critical when you want to interface Erlang to the outside world: if your C library contains a routine that might block for a long time, you can't just call it from Erlang as it won't be a well-behaved part of Erlang's world (there are ways around this of course, but make life a bit more complicated for the developer).  All this is done with Erlang's scheduler: each Erlang process gets a number of operations it can run before some other process gets to run, so even our while loop will only run for a bit before the system moves on to something else.   IO is rigorously done with non-blocking calls internally in order to keep that from becoming an issue.

No other system that I know of has such a focus on being non-blocking, and node.js is no exception: a while(true) loop is perfectly capable of wedging the system.  Node.js works by passing functions (closures, in many cases) around so that work can be performed as needs be.  However, the actual functions that run actually do block the system, and thus must be written in order to not run for too long.  Also, and this is important, Node.js also does its best to make IO non-blocking by default, so that you don't have to worry about IO calls.

Node.js isn't up to the level Erlang is at, because it requires more manual intervention and thinking about how to do things, but it's probably "good enough" for many tasks.  How often do you really write code with so many calculations that it slows things down?  Not often in my case – the real world problem I most often encounter is IO, and node.js does its best to make that non-blocking, so that it can be handled in the "background" or a bit at a time, without wedging the system.  And if you really needed to write a long-running calculation (say you want to stream the digits of PI or something), you can break up your calculation manually, which may not be quite as elegant as Erlang, but is "good enough" for many people.

"Good enough" concurrency, combined with a language that is at least an order of magnitude more popular than Erlang, and a fast runtime, combined with ease of use in general (it's way easier to get started with node.js than with most Erlang web stuff) make for a system that's likely to do fairly well in terms of diffusion and popularity, and is going to "eat some of Erlang's lunch".  Or perhaps, rather than actually taking users away from Erlang, it's likely to attract people that might have otherwise gone to Erlang.

Javascript Charts

I’ve been doing some work on LangPop.com, and one of the things I’d like to do is update the chart software. What I have now, Plotr seems to work ok, but being the tinkering type, I want to fix things, even if they’re not broke. Truth be told, my worry is that Plotr isn’t maintained any more, so it would be a good idea to find something that’s receiving a bit of attention.

The candidates:

These seem to be fairly up to date in the sense that someone worked on them recently. There are some older ones like PlotKit, that do not appear to be maintained any more. It is entirely possible that I missed a good one. Another possibility would be to use Google’s chart API, but I’d rather be a little bit more in control of things than to farm that out, and I also am planning on doing some interactive features in the near(ish) future.

So let’s have a look:

Flot

Based on JQuery, this one looks fairly complete, and has a lot of different, nice looking charts. Since I don’t really care what library it’s based on JQuery seems as good as any other, being quite popular these days. Installation is pretty simple, and, having good defaults, it’s easy to get nice looking data up on the screen in short order.

Flotr

Modeled after Flot, Flotr uses Prototype (the Rails default) instead of JQuery, and is the follow up to Plotr. I’m not quite sure what the motivation was behind the name/project change, but this seems to be where the guy is spending his time. Since he did a good job with Plotr, this one seemed worth a look too.

ProtoChart

This one is based on Prototype too. It immediately annoyed me by uncompressing from the .zip file in the current directory, scattering files around. I’ve always found this behavior a bit antisocial. This code claims to be “motivated by Flot, Flotr and PlotKit libraries”, which indicate that it’s fairly recent. However, my feeling is that the problem is not necessarily how old code is in these projects, but how quickly they spring up, bloom, and then stagnate. I’d like to use something that’s got some staying power… But anyway, ProtoChart looks like pretty good code, even if the distribution is a bit minimal, and doesn’t come with examples.

If you look at LangPop.com, you’ll notice something that’s very important for my choice: I need a barchart that has horizontal bars, rather than the more traditional vertical bars. This is because it would be quite difficult to squeeze so many languages across the screen horizontally. This ended up being the deciding factor: like Plotr, Flotr supports horizontally oriented bar charts, making it the obvious choice. The other two libraries looked pretty good too – Flot, in particular, looks like quite solid code.

So, there you have it – very brief reviews of an incomplete selection of libraries! In my defense, the goal is to attempt to dominate my maximizing nature, pick one, and get on with doing some cool stuff with LangPop. I do, however, welcome comments and suggestions.

YUI Compressor

One of the things I love about programming is that you are always learning new things. You have to learn new things to keep up.

One of the benefits of using the Plotr Javascript library for the language popularity stats I did is that instead of having to download separate images for each statistic, a user just fetches the library once and then a few small bits of data for each chart, which is probably a net savings, especially when I start to add more statistics. However, Plotr, plus its various supporting libraries, is fairly hefty, so I needed to find a way to compress it. One of the helpful people on the freenode IRC network’s #javascript channel pointed me at Yahoo’s YUI Compressor, which is a nicely done project that squeezes quite a bit out of Javascript files. I would recommend it to anyone else who needs to pare down their production JS files to the bare minimum. Best of all, it’s available under a very liberal open source license!

Incidentally, over the course of the language stats project, I was surprised at how much helpful stuff Yahoo provides, first and foremost, a real API for their search engine, something that Google no longer has. Hopefully, it’s a sign of better things to come for them (if for no other reason than to keep Google on their toes!).

By the way, for those interested in the language stats project, I set up a forum with RSS feeds, where I will be posting updates, and accepting suggestions for further improvements.

Chartr – Plotr generation for Rails

I’ve been working on a project that uses graphics created with Plotr, and I wanted to automate that from Rails, so I created Chartr. It’s not very complete, but works well enough to create stuff like this:


<script src="http://chartr.rubyforge.org/excanvas.js" type="text/javascript"></script>
<script src="http://chartr.rubyforge.org/plotr_uncompressed.js" type="text/javascript"></script>
  <canvas id="results_chart" height="300" width="500"></canvas>
</div>

var dataset = {“Number/Decade”: [[0, 1000], [1, 5000], [2, 20000], [3, 22000]]};
var options = {colorScheme: “#22aa22”, legend: {position: {}}, axis: {x: {ticks: [{label: “1960”, v: 0}, {label: “1970”, v: 1}, {label: “1980”, v: 2}, {label: “1990”, v: 3}]}}, padding: {left: 100}, background: {}, stroke: {}, legend: {position: {left: ‘120px’}}};
var discvar = new Plotr.BarChart(“results_chart”, options);
discvar.addDataset(dataset);
discvar.render();


Help is more than welcome! I think the API is still a bit clumsy, more chart types need adding, and there are other things that could probably be improved, but it does work.