Rails - Aug 2006 - Article: High-Performance Ruby On Rails Setups Test...

This week we have started one new project with Ruby on Rails as primary 
framework. My first task was to prepare runtime environment for it on 
one of our development servers. When I have tried to research how people 
doing it, I noted that there is no information about how to deploy rails 
application with nginx as frontend and what is performance of such 
solution. Before blindly make any decisions about future platform I’ve 
decided to make some performance tests of the some most popular rails 
web servers/frontends. Results of these tests you can find here with 
configuration samples for all software that I have used.

Details: 
http://blog.kovyrin.net/2006/08/22/high-performance-rails-nginx-lighttpd-mongrel

-- 
Posted via http://www.ruby-forum.com/.

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk
-~----------~----~----~----~------~----~------~--~---

Alexey Kovyrin wrote:
> This week we have started one new project with Ruby on Rails as primary 
> framework. My first task was to prepare runtime environment for it on 
> one of our development servers. When I have tried to research how people 
> doing it, I noted that there is no information about how to deploy rails 
> application with nginx as frontend and what is performance of such 
> solution. Before blindly make any decisions about future platform I’ve 
> decided to make some performance tests of the some most popular rails 
> web servers/frontends. Results of these tests you can find here with 
> configuration samples for all software that I have used.
>
> Details: 
>
http://blog.kovyrin.net/2006/08/22/high-performance-rails-nginx-lighttpd-mongrel
>   
First of all - great initiative. But - as is posted in the comments on 
your blog - testing the default rails index.html page really isn''t 
performance testing rails at all - since that''s a static html page. You
need to create an app that probably hits a database on every page view 
or at least hits and then caches.

I''m also questioning how you came up with testing 100 simultaneous 
connections from one single computer. I''m sure the tcp/ip stack of the 
OS in question will be a factor here - and 100 connections 
simultaneously is a LOT.



--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk
-~----------~----~----~----~------~----~------~--~---

As they''ve already told you, there''s a need to make the same
tests with
some kind of DB query... Here are my results:

Test machine: Sun Fire X4100 2xOpteron 248, Debian 3.1, kernel 
2.6.8-smp, 2GB RAM

Servers tested:
- Webrick 1.3.1
- Lighttpd 1.4.11 + 5 fcgi spawns
- Mongrel 0.3.13.3
- Pound & 5xMongrel cluster
(apache will be included in the next round)

The test:
- New rails application, the database is a list of the results of a 
lottery game in my country (Kino). Modified the ''list'' action
of the
scaffold-created controller to display all the 350 rows. The resulting 
page is about 500Kb. There''s no cache, the environment is production.
The log was erased after every test.

The benchmark:
Three different tests:
T1: 20 concurrent connections, 500 total requests
T2: 30 concurrent connections, 500 total requests
T3: 40 concurrent connections, 500 total requests

The command:
/usr/sbin/ab -c $CONN -n $TOTAL http://cerebro:3000/sorteos
Run from another machine in the LAN.

The results:
The test was run 10 times. Results here are the average.

Webrick (only T1)
T1:
  - Total:    460.43 [s]
  - Req/s:    1.09
  - Rate:     570.63 [s]
  - Time/req: 920.00 [ms]

Lighttpd + 5 fcgi spawns:
T1:
  - Total:    207.15 [s]
  - Req/s:    2.41
  - Rate:     1268.18 [s]
  - Time/req: 414.31 [ms]
T2:
  - Total:    216.23 [s]
  - Req/s:    2.31
  - Rate:     1214.93 [s]
  - Time/req: 432.47 [ms]
T3:
  - Total:    215.67 [s]
  - Req/s:    2.32
  - Rate:     1218.07 [s]
  - Time/req: 431.35 [ms]

Mongrel (only T1)
T1:
  - Total:    540.90 [s]
  - Req/s:    0.92
  - Rate:     486.05 [s]
  - Time/req: 1081.82 [ms]

Pound + 5xMongrel Cluster
T1:
  - Total:    290.40 [s]
  - Req/s:    1.72
  - Rate:     908.77 [s]
  - Time/req: 580.81 [ms]
T2:
  - Total:    324.54 [s]
  - Req/s:    1.54
  - Rate:     813.96 [s]
  - Time/req: 649.07 [ms]

In my scenario, the best option appears to be lighttpd + fcgi.

Regards,
Rolando.-

-- 
Posted via http://www.ruby-forum.com/.

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk
-~----------~----~----~----~------~----~------~--~---

On Wed, 2006-08-23 at 14:50 +0200, Rolando Abarca wrote:
> As they''ve already told you, there''s a need to make the
same tests with
> some kind of DB query... Here are my results:
You also have to run each test multiple times and take the mean of them
in order to rule out accidentally getting a bad read.  Not to mention a
bunch of other stuff.

<modified below>
> Webrick (only T1)
> T1:  - Req/s:    1.09
> Lighttpd + 5 fcgi spawns:
> T1:  - Req/s:    2.41
> T2:  - Req/s:    2.31
> T3:  - Req/s:    2.32
> Mongrel (only T1)
> T1:  - Req/s:    0.92
> Pound + 5xMongrel Cluster
> T1:  - Req/s:    1.72
> T2:  - Req/s:    1.54
All of these are so pathetically low that I seriously doubt your test is
configured correctly.  These are so low that you''re either running this
against a worst case setup or in development mode or something.  In
fact, these numbers are so low it doesn''t matter how fast your setup
is,
it can''t service any kind of reasonable requests rate.

In your situation there''s probably so many confounding factors that
you''d need to work on the setup to find the simplest best case as a
comparison (like the control in an experiment).  Until you do that you
really have no idea how fast each config could be.

For example, how is it possible that you have such a low performance and
yet with 5 mongrels or fcgi processes you don''t see a 5x increase?
That''s the expected response.  Also, why 5?  How come there''s
no test
for lighttp+1 fcgi?  Is this with everything running on the same machine
(even your ab run)?  How many of these runs did you do?  Just one?  How
do you know that one wasn''t just bad/good luck?  Why are you increasing
the concurrency but not the number of connections?

Just some minor criticisms as I don''t think the test is right for any
of
the tested servers (I *know* fcgi does better than 2.3 req/s, as well as
mongrel).


-- 
Zed A. Shaw
http://www.zedshaw.com/
http://mongrel.rubyforge.org/
http://www.lingr.com/room/3yXhqKbfPy8 -- Come get help.


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk
-~----------~----~----~----~------~----~------~--~---

Zed Shaw wrote:
> You also have to run each test multiple times and take the mean of them
> in order to rule out accidentally getting a bad read.  Not to mention a
> bunch of other stuff.
quoting myself:

  > The results:
  > The test was run 10 times. Results here are the average.
> All of these are so pathetically low that I seriously doubt your test is
> configured correctly.  These are so low that you''re either running
this
> against a worst case setup or in development mode or something.  In
> fact, these numbers are so low it doesn''t matter how fast your
setup is,
> it can''t service any kind of reasonable requests rate.
The rendered page is a 500Kb page. This is the controller code:

  def list
    #@sorteo_pages, @sorteos = paginate :sorteos, :per_page => 10
    @sorteos = Sorteo.find(:all)
  end

The view code is the one generated by the scaffold, minus the pagination 
stuff.
> In your situation there''s probably so many confounding factors
that
> you''d need to work on the setup to find the simplest best case as
a
> comparison (like the control in an experiment).  Until you do that you
> really have no idea how fast each config could be.
I could do the same test our friend Alexey did: hit the index page.
> For example, how is it possible that you have such a low performance and
> yet with 5 mongrels or fcgi processes you don''t see a 5x increase?
> That''s the expected response.  Also, why 5?  How come
there''s no test
> for lighttp+1 fcgi?
Yes... I should''ve tested lighttpd+1 fcgi.
> Is this with everything running on the same machine
> (even your ab run)?
again, quoting myself:

  > The command:
  > /usr/sbin/ab -c $CONN -n $TOTAL http://cerebro:3000/sorteos
  > Run from another machine in the LAN.
> How many of these runs did you do?  Just one?  How
> do you know that one wasn''t just bad/good luck?  Why are you
increasing
> the concurrency but not the number of connections?
Again, each test was run 10 times. The results in the post are the 
average. Also, you have to consider that the rendered page was about 
500Kb. There were no images, no external stylesheets, etc.
> Just some minor criticisms as I don''t think the test is right for
any of
> the tested servers (I *know* fcgi does better than 2.3 req/s, as well as
> mongrel).
Did you read the *whole* post? I was trying to hit a worst case 
scenario, not an average one. That''s why I removed the pagination,
there
was no cache (except the default in the production environment), that''s
why I rendered a whole ~350 rows html-table to the browser.
I might run the same test, but this time with pagination (to reduce the 
size of the result for the query and the rendered page)...
> --
> Zed A. Shaw
Regards,
rolando.-

-- 
Posted via http://www.ruby-forum.com/.

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk
-~----------~----~----~----~------~----~------~--~---

Zed Shaw wrote:
> Just some minor criticisms as I don''t think the test is right for
any of
> the tested servers (I *know* fcgi does better than 2.3 req/s, as well as
> mongrel).
oh!... one more thing. Don''t check the req/s, check to see if the rate 
is according your results.
The request I made is not a standard one, so that might be the reason of 
the low req/s, but check for the rate.
(btw, the unit for the rate I put in my post is [KB/s], not [s]...)
> --
> Zed A. Shaw
regards,
Rolando.-

-- 
Posted via http://www.ruby-forum.com/.

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk
-~----------~----~----~----~------~----~------~--~---

On Thu, 2006-08-24 at 15:21 +0200, Rolando Abarca wrote:
> Zed Shaw wrote:
> Did you read the *whole* post? I was trying to hit a worst case 
> scenario, not an average one. That''s why I removed the pagination,
there
> was no cache (except the default in the production environment),
that''s
> why I rendered a whole ~350 rows html-table to the browser.
> I might run the same test, but this time with pagination (to reduce the 
> size of the result for the query and the rendered page)...
Yes I read it, I was just giving you some friendly advice on how to do
it again so that it actually showed something.  Your worst case scenario
is basically making it so that your performance measurement has no
information.  You''ve confounded the analysis with all the extra
conditions you put on it, when it''s not really necessary.

For example, how do I know your "worst case scenario" isn''t
really:

  def sleep_for_days
    sleep 60
  end

I don''t, so without a baseline to compare with on the same machine I
have nothing to go on (unless you post your code).  What if you run your
baseline and come up with say 20 req/sec?  Well, I''d assume your system
is setup poorly since I can get much smaller machines to do better than
that.  Without a baseline, I have nothing to compare against.

Now I understand it was done quickly, but if you post measurements like
that without taking the time to do them right then it just gives
everyone a bunch of FUD to deal with.  A better approach would be to
have done all your analysis for the baseline fastest case (as a control)
*then* your analysis.

Also, don''t just say X is faster than Y.  Really clarify it, "With
a
blah controller that has a blah condition with blah database and no
caching on foo machine, then X is faster than Y."

Because, just like the nightly news, all people will post on their blogs
is:

  "X is faster than Y".

Which does us all a disservice in the end.

-- 
Zed A. Shaw
http://www.zedshaw.com/
http://mongrel.rubyforge.org/
http://www.lingr.com/room/3yXhqKbfPy8 -- Come get help.


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk
-~----------~----~----~----~------~----~------~--~---

Zed Shaw wrote:
> Now I understand it was done quickly, but if you post measurements like
> that without taking the time to do them right then it just gives
> everyone a bunch of FUD to deal with.  A better approach would be to
> have done all your analysis for the baseline fastest case (as a control)
> *then* your analysis.
Ok, I give you that. I''m preparing a baseline test.
> Also, don''t just say X is faster than Y.  Really clarify it,
"With a
> blah controller that has a blah condition with blah database and no
> caching on foo machine, then X is faster than Y."
I never explicitly said "X is faster than Y", I actually said "in
my
scenario, the best option appears to be lighttpd + fcgi". There''s
an
"appears" and "in my scenario".
> Because, just like the nightly news, all people will post on their blogs
> is:
> 
>   "X is faster than Y".
> 
> Which does us all a disservice in the end.
I totally agree with you.
> --
> Zed A. Shaw
> http://www.zedshaw.com/
> http://mongrel.rubyforge.org/
> http://www.lingr.com/room/3yXhqKbfPy8 -- Come get help.
regards,
rolando.-

-- 
Posted via http://www.ruby-forum.com/.

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk
-~----------~----~----~----~------~----~------~--~---

Alexey Kovyrin wrote:
> I noted that there is no information about how to deploy rails 
> application with nginx as frontend and what is performance of such 
> solution. 
The day after you wrote this, Ezra Zygmuntowicz posted this on his blog:

"Nginx, my new favorite front end for mongrel cluster"
http://brainspl.at/articles/2006/08/23/nginx-my-new-favorite-front-end-for-mongrel-cluster

regards

   Justin

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk
-~----------~----~----~----~------~----~------~--~---

>
>
> "Nginx, my new favorite front end for mongrel cluster"
>
>
http://brainspl.at/articles/2006/08/23/nginx-my-new-favorite-front-end-for-mongrel-cluster
>
>
Thanks for the reference Justin,
-andy


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk
-~----------~----~----~----~------~----~------~--~---