[Mongrel] Mongrel performance study

Steve Midgley
2008-04-08T21:36:16Z

>On Mon, 07 Apr 2008 23:14:47 -0700
> >
> >
>
http://www.misuse.org/science/2008/04/07/thin-ruby-on-rails-nginx-fair-proxy-performance-testing/
>
>Yay! You mentioned standard deviations.  Bad! You didn''t give any.
>That''d help people figure out if the 4% is even worth it.

I did! I did! The raw data (including std dev''s) were linked off the
main post. That link is:
http://www.misuse.org/downloads/thin_mongrel_test_results.html - I
thought that level of info on the main post would be too
distracting/confusing for many readers..

I tried to be meticulous in collecting these data, and Zed, for every
std dev cut and paste out of JMeter I thought off you. :)

>[snip]
>"Thin is the newest incarnation of Zed Shaw''s (and now
community
>managed) masterwork Mongrel."
>
>That statement isn''t correct.  Please don''t make it since
I don''t
>think
>the author of Thin or the authors of Mongrel would say that.  Oh, well
>they''d say my stuff is a masterwork, just not that Thin is an
>incarnation of Mongrel. :-)

Ok - that''s fixed. I said that b/c I thought Thin is using
your/mongrel''s ragel http states. But you know (way) more than me on
this. So I changed the language.

>[snip]
>I don''t think it''s unix sockets vs. ip sockets.  These
days those
>aren''t really that much faster than a localhost connection thanks
to
>advancements in performance for internal IPC.

The Thin author Marc said the same thing. I''ll adjust that in the
article soon.

>[snip]
>Another comment I''d make is, why not also release the methodology
you
>gave.  I''m actually working on a presentation for RuPy so if
you''d
>like
>to hack on a repeatable study with an automated report let me know.
>Could be a good test for what I''m writing for the conference.

Thanks for the encouragement. I just published my methodology, which
was the same for both tests, at the end of the second report:
http://www.misuse.org/science/2008/04/07/thin-vs-mongrel-a-ruby-on-rails-performance-shootout/

Feedback and criticism are welcome on that methodology. I have done a
fair bit of performance testing in the past, but not in such a general
sense as comparing two different application servers, so I may have
done something wrong.

I''d also be open to performing more tests using this methodology and
codebase, but I don''t have access to an unloaded high performance
webserver anymore. I could probably talk Ezra or Wayne into loaning me
one though. :)

Drop me a line here or personally if that sounds useful.

Steve