Fresh New Ruby Implementation Benchmarks: So Who’s Fastest?
Almost a year ago, Antonio Cangiano performed some benchmarks on the then present Ruby implementations: Ruby 1.8.5, YARV (now Ruby 1.9), JRuby, Ruby.NET, Rubinius and Cardinal. The results were that YARV, although nowhere near ready for production use, was streets ahead of Ruby 1.8, and the other implementations raised enough errors to still be considered 'experimental'.
Now Antonio has rerun the tests on Ruby 1.8.6, Ruby 1.9 (from trunk), JRuby (from trunk), Rubinius (from trunk) and XRuby 0.3.2. Ruby 1.9 again takes the lead by quite a margin, but the best development is that instead of being a distant third (behind Ruby 1.8), JRuby has made significant improvements and is smack bang in between Ruby 1.8 and Ruby 1.9 in terms of performance (while offering, of course, the benefits of the Java ecosystem). JRuby now also passes all of the tests without error, a feat that even Ruby 1.9 doesn't yet achieve. The Rubinius team hasn't been slacking either. On the previous test, Rubinius was coming in miles behind the other implementations, but is now quicker than the stock Ruby 1.8.6 interpreter on a wide selection of the tests.
Antonio's post is blog pornography for those of us who are into benchmarks, comparisons, and graphs, so just go soak it all up now.
December 4, 2007 at 12:50 am
It seems Antonio's blog couldn't handle the load, was... Insided! ;-)
Will be great if the bench aren't those from ruby1.9 trunk since most of them tuned for it.
what a general sudoku solver or something like that? :-)
December 4, 2007 at 4:04 am
From Antonio's blog: "None of the tests took startup times into consideration." May I ask why the startup time is not considered?
December 4, 2007 at 4:43 am
Between the list of 1.9 changes (http://www.dzone.com/links/rss/changes_in_ruby_19.html) and the blazing speed it has to offer I am chomping at the bit for it to drop.
I played with the trunk a little bit but the application I'm working on right now does the heavy lifting in C and 1.9 actually handled exceptional conditions in C poorly (read: felt like a fork bomb) whereas 1.8.x at least gives me a Bus Error + Abort Trap and saves my CPU from frying.
Also a little miffed at losing my inherited class variables but no biggie, just shows that I was using them outside the scope that Matz intended.
Bring on the 1.9!
December 4, 2007 at 3:04 pm
Francois G: One reason might be that JRuby has a glacial startup time compared to the regular Ruby interpreter (MRI) but its performance, once up, is good.
As many applications now run in a service like fashion (such as Rails apps), the speed of the interpreter, rather than its overheads, is more important.