Rubyinstaller devel - Nov 2008 - Some benchmarks with MinGW, JRuby, MSVC...

So, I have managed to build several different versions of Ruby to compare all
in some benchmarks. I am going to let the numbers talk for themselves. It was
all tested on an old desktop computer with Windows XP, so I am not testing
anything too impressive.

Of importance is the seemingly poor performance of Ruby 1.8.7 for whatever
reason. It''s as if Ruby 1.8.7 were like Windows ME, a version between
major
versions, as an analogy. :-)

Also, I was not able to run the Mongrel gem on Ruby 1.9.1.

I am going to order the data by fastest to make it easier to compare it on text.
I have collected the data on a Google Spreadsheet:
http://spreadsheets.google.com/ccc?key=pmtaDyNoButGKeSVdIGi7Fg

I am going to keep it short here and for more detail on versions and whatnot
please refer to the spreadsheet.

== WEBrick (req/s)

22.39 - jruby
16.95 - jruby --server
12.82 - ruby oneclick 1.8.6
12.49 - ruby msvc 1.8.6
12.45 - ruby rubyinstaller 1.8.6
12.43 - ruby mingw 1.8.6
11.94 - ruby msvc 1.9.1
11.48 - ruby mingw 1.9.1
3.42 - ruby msvc 1.8.7
3.40 - ruby mingw 1.8.7


== Mongrel (req/s)

357.14 - ruby mingw 1.8.6
347.45 - ruby rubyinstaller 1.8.6
326.70 - ruby msvc 1.8.6
313.57 - ruby oneclick 1.8.6
274.68 - ruby mingw 1.8.7
232.39 - ruby msvc 1.8.7
71.24 - jruby
62.22 - jruby --server
(crash when starting up) - ruby mingw 1.9.1
(error when building gem) - ruby msvc 1.9.1


== bench_pythag.rb (s)

19.09 - jruby --server
24.35 - jruby
31.72 - ruby mingw 1.9.1
31.91 - ruby mingw 1.8.6
32.73 - ruby mingw 1.8.7
33.94 - ruby msvc 1.9.1
35.67 - ruby rubyinstaller 1.8.6
35.83 - ruby msvc 1.8.7
39.79 - ruby msvc 1.8.6
48.30 - ruby oneclick 1.8.6


== bench_fractal.rb (s)

3.22 - jruby --server
3.89 - jruby
5.67 - ruby msvc 1.9.1
5.75 - ruby mingw 1.9.1
9.80 - ruby mingw 1.8.6
9.90 - ruby mingw 1.8.7
10.11 - ruby rubyinstaller 1.8.6
10.30 - ruby msvc 1.8.6
10.64 - ruby msvc 1.8.7
15.17 - ruby oneclick 1.8.6


== bench_float_math.rb (s)

103.44 - jruby --server
125.89 - jruby
158.64 - ruby msvc 1.9.1
175.41 - ruby mingw 1.9.1
176.61 - ruby mingw 1.8.6
179.94 - ruby mingw 1.8.7
184.73 - ruby rubyinstaller 1.8.6
195.37 - ruby msvc 1.8.7
197.39 - ruby msvc 1.8.6
279.62 - ruby oneclick 1.8.6


Since I collected these data, the Ruby 1.9.1 branch has had a number of updates
for one thing. :-)

Benchmarks can be kind of tough to do right and there are all
possibilities when it
comes to doing them.

If anyone has an algorithm that he would like to test on all those
Ruby implementations
I would be glad to test it. With the exception of testing Rails, which
I have never used
and don''t know enough about it to willingly make myself available when
testing it. :-)

Cheers,
Joao