Ruby Shootout
Antonio Cangiano has an interesting set of Ruby benchmarks here:
http://antoniocangiano.com/2008/12/09/the-great-ruby-shootout-december-2008/
The two results that were interesting to me were that JRuby does so well, and that REE ("Ruby Enterprise Edition") was a significant improvement over "plain old Ruby" too. I knew that the JRuby guys have done a lot of work, and done a nice job with their implementation, but didn't realize just how good it was. REE is a clever idea, but not that much code, and aimed at memory usage, not CPU, so it's interesting to see how much faster it ends up according to these tests.
A few things that would be interesting to see:
Raw startup time. Yeah, that probably penalizes JRuby, but it is still a scripting language, and using it like one isn't possible if it takes forever to start up. How much slower is it?
A more detailed look at REE's times. How is it that it's that much faster because of some memory tweaks?
In any case, it's a nice bit of work by Antonio - grazie!
Update - old, slow davidw missed the fact that there was a startup time benchmark included. Oops.
Trackbacks
Use the following link to trackback from your own site:
http://journal.dedasys.com/trackbacks?article_id=2118
about 2 hours later:
The result does include raw startup time.
about 4 hours later:
Reducing memory usage is fully expected to improve performance significantly. These days memory access is so much slower than CPU, so reducing memory usage leads to bigger part of the core working set to fit in the caches which means a lot fewer slow misses.
Actual improvement will vary widely on the actual memory access pattern, though.
about 5 hours later:
Jan, sure, that's to be expected, but consider that the algorithm running is probably running the exact same number of Ruby objects, and I don't think those are any smaller: they are some kind of struct, and a nice allocator isn't going to change how big they are. Perhaps it's the speed of tcmalloc rather than any actual memory savings.
10 days later:
Antonio has an update where he talks about his discover (and talks with the REE guys) that his stock binary install of Ruby was probably not so good. It's fairly well known that Ruby is very sensitive to compiler optimizations, and almost everyone with a binary dist has chosen the wrong set and killed performance. When he did a second pass with ruby-from-source, it performed much better.
Of course, JRuby just distributes binary and relies on the JVM to do all optz. I think JRuby's the way to go, but I'm biased :)
And to answer your question: JRuby's startup time ranges from 0.4s (on my Mac) to a couple seconds (on some Linux boxes). There's ways to improve it, but we're somewhat bound by the speed at which the JVM can load all the JRuby code along with the Java libraries we access. So yeah, startup time isn't great, but you can look past that when your script ends up running a lot faster. We make up the difference.