Mongrel users - Feb 2007 - Mongrel performing only half as fast as Apache?

I''m trying to do some initial benchmarking of our setup, mainly just to
establish baselines.  I''m essentially using the process Zed outlines in
a
previous message:
http://rubyforge.org/pipermail/mongrel-users/2006-May/000200.html

What I''m running into is that Mongrel appears only half as fast as
Apache
when serving a small static HTML file.  If I then add in Apache with
mod_proxy_balancer, going to a single Mongrel, it drops down to nearly about
a third of what pure static Apache will do.  This seems bogus to me, and I
suspect I have either some configuration problem, or something.  My
understanding from what I''ve read is that Mongrel should be fairly
close to
Apache when serving static content (at least not only 50% as fast).  Is that
right as a generalization?

Here''s some info to back this up.
- Server: HP DL360 dual 3.0GHz Xeons, 4GB RAM, 10k RPM SCSI disks
- OS: Fedora Core 6, up to date as of a few days ago, kernel 2.6.18 smp
- Apache 2.2.3, with mod_proxy_balancer (the RPM install)
- Mongrel 1.0.1, mongrel_cluster 0.2.1
- ruby 1.8.5 (2006-12-04 patchlevel 2)
- serving a static HTML file from the Rails app''s public directory
- Testing using an identical server, that sits above this one in the rack,
connected to a switch (so it''s one hop)
- Using httperf for testing, with the following command:
  httperf --server lab05 --port 80 --uri /mongrel_alive.html --num-conns
10000
The number of connections I vary to be near the 10 second mark...

Here''s the results:

- Just Apache, num-conns=15000, ~1400 req/sec
- Direct to Mongrel on port 5000, num-conns=8000, ~740 req/sec
- Apache mod_proxy_balancer to a single Mongrel, num-conns=5000, ~475
req/sec

So, I''m incurring a massive penalty for the balancer/cluster setup.  In
production we will be using hardware load balancers, but even still, my
understanding was that instead of a drop from 1400 to 740, it should be
somewhat closer (say at least over 1000).

My configuration is what seems to be the usual thing documented say in Rails
Cookbook, on various blogs, and so on.  I can post the details if needed.

What would folks suggest, or what comments do you have?
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/mongrel-users/attachments/20070227/67e11a05/attachment.html

As additional info, I ran the same kind of tests with Pen and Nginx instead
of Apache+mod_proxy_balancer, just to get an idea on that aspect:

Pen with 1 Mongrel: ~671 req/sec
Nginx with 1 Mongrel: ~612 req/sec

So these at least show a smaller overhead for the load balancer aspect.
They also are more bare-minimum configured (I don''t have much
experience
with either of those, so just did the basic Pen setup where you tell it what
port to run on and the ports/domain+port it''s proxying).

On 2/27/07, Christopher Bailey <chris at codeintensity.com>
wrote:
>
> I''m trying to do some initial benchmarking of our setup, mainly
just to
> establish baselines.  I''m essentially using the process Zed
outlines in a
> previous message:
> http://rubyforge.org/pipermail/mongrel-users/2006-May/000200.html
>
> What I''m running into is that Mongrel appears only half as fast as
Apache
> when serving a small static HTML file.  If I then add in Apache with
> mod_proxy_balancer, going to a single Mongrel, it drops down to nearly
about
> a third of what pure static Apache will do.  This seems bogus to me, and I
> suspect I have either some configuration problem, or something.  My
> understanding from what I''ve read is that Mongrel should be fairly
close to
> Apache when serving static content (at least not only 50% as fast).  Is
that
> right as a generalization?
>
> Here''s some info to back this up.
> - Server: HP DL360 dual 3.0GHz Xeons, 4GB RAM, 10k RPM SCSI disks
> - OS: Fedora Core 6, up to date as of a few days ago, kernel 2.6.18 smp
> - Apache 2.2.3, with mod_proxy_balancer (the RPM install)
> - Mongrel 1.0.1, mongrel_cluster 0.2.1
> - ruby 1.8.5 (2006-12-04 patchlevel 2)
> - serving a static HTML file from the Rails app''s public directory
> - Testing using an identical server, that sits above this one in the rack,
> connected to a switch (so it''s one hop)
> - Using httperf for testing, with the following command:
>   httperf --server lab05 --port 80 --uri /mongrel_alive.html --num-conns
> 10000
> The number of connections I vary to be near the 10 second mark...
>
> Here''s the results:
>
> - Just Apache, num-conns=15000, ~1400 req/sec
> - Direct to Mongrel on port 5000, num-conns=8000, ~740 req/sec
> - Apache mod_proxy_balancer to a single Mongrel, num-conns=5000, ~475
> req/sec
>
> So, I''m incurring a massive penalty for the balancer/cluster
setup.  In
> production we will be using hardware load balancers, but even still, my
> understanding was that instead of a drop from 1400 to 740, it should be
> somewhat closer (say at least over 1000).
>
> My configuration is what seems to be the usual thing documented say in
> Rails Cookbook, on various blogs, and so on.  I can post the details if
> needed.
>
> What would folks suggest, or what comments do you have?
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/mongrel-users/attachments/20070227/fe0e762c/attachment.html

On 2/27/07, Christopher Bailey <chris at codeintensity.com> wrote:
> What I''m running into is that Mongrel appears only half as fast as
Apache
> when serving a small static HTML file.  If I then add in Apache with
> mod_proxy_balancer, going to a single Mongrel, it drops down to nearly
about
> a third of what pure static Apache will do.  This seems bogus to me, and I
> suspect I have either some configuration problem, or something.  My
> understanding from what I''ve read is that Mongrel should be fairly
close to
> Apache when serving static content (at least not only 50% as fast).  Is
that
> right as a generalization?
That may not be far off at all, and that difference between going
directly to Mongrel versus going through Apache with proxying that you
report is about what I have seen.
> - Just Apache, num-conns=15000, ~1400 req/sec
That actually seems really slow.  My box is a lot slower than yours
(dual 2ghz 32 bit AMD processors on 2gb RAM, older kernel, slower
disk), and I get twice that speed through Apache (2.0) to a small
static file.  About 2800/second.
> - Direct to Mongrel on port 5000, num-conns=8000, ~740 req/sec
This is close to what I have seen in my direct-to-mongrel tests,
though I get speeds that are a little bit higher.  In the 800+
req/second range.  I don''t think anything is wrong with your tests.


Kirk Haines

Christopher,

Mongrel is not meant to serve static files. It is meant to replace  
the FCGI part of doing Apache->FCGI->Ruby. Or WhateverBalancer->FCGI- 
 >Ruby.

As Ezra put its here, http://f8p.com/96ktde , "...it is horribly  
ineficient to serve static files with mongrel..."

Most mongrel setups (ones with Apache) use mod_rewrite to server the  
static files through apache (including cached rails html files).

Try this vhost template ( http://f8p.com/6d6i0s ) that uses  
mod_rewrite to serve static HTML through Apache.

-christos

On 28 Feb 2007, at 01:48, Christopher Bailey wrote:
> As additional info, I ran the same kind of tests with Pen and Nginx  
> instead of Apache+mod_proxy_balancer, just to get an idea on that  
> aspect:
>
> Pen with 1 Mongrel: ~671 req/sec
> Nginx with 1 Mongrel: ~612 req/sec
>
> So these at least show a smaller overhead for the load balancer  
> aspect.  They also are more bare-minimum configured (I don''t have
> much experience with either of those, so just did the basic Pen  
> setup where you tell it what port to run on and the ports/domain 
> +port it''s proxying).
>
> On 2/27/07, Christopher Bailey <chris at codeintensity.com> wrote:
I''m
> trying to do some initial benchmarking of our setup, mainly just to  
> establish baselines.  I''m essentially using the process Zed  
> outlines in a previous message:
> http://rubyforge.org/pipermail/mongrel-users/2006-May/000200.html
>
> What I''m running into is that Mongrel appears only half as fast as
> Apache when serving a small static HTML file.  If I then add in  
> Apache with mod_proxy_balancer, going to a single Mongrel, it drops  
> down to nearly about a third of what pure static Apache will do.   
> This seems bogus to me, and I suspect I have either some  
> configuration problem, or something.  My understanding from what  
> I''ve read is that Mongrel should be fairly close to Apache when  
> serving static content (at least not only 50% as fast).  Is that  
> right as a generalization?
>
> Here''s some info to back this up.
> - Server: HP DL360 dual 3.0GHz Xeons, 4GB RAM, 10k RPM SCSI disks
> - OS: Fedora Core 6, up to date as of a few days ago, kernel 2.6.18  
> smp
> - Apache 2.2.3, with mod_proxy_balancer (the RPM install)
> - Mongrel 1.0.1, mongrel_cluster 0.2.1
> - ruby 1.8.5 (2006-12-04 patchlevel 2)
> - serving a static HTML file from the Rails app''s public directory
> - Testing using an identical server, that sits above this one in  
> the rack, connected to a switch (so it''s one hop)
> - Using httperf for testing, with the following command:
>   httperf --server lab05 --port 80 --uri /mongrel_alive.html --num- 
> conns 10000
> The number of connections I vary to be near the 10 second mark...
>
> Here''s the results:
>
> - Just Apache, num-conns=15000, ~1400 req/sec
> - Direct to Mongrel on port 5000, num-conns=8000, ~740 req/sec
> - Apache mod_proxy_balancer to a single Mongrel, num-conns=5000,  
> ~475 req/sec
>
> So, I''m incurring a massive penalty for the balancer/cluster  
> setup.  In production we will be using hardware load balancers, but  
> even still, my understanding was that instead of a drop from 1400  
> to 740, it should be somewhat closer (say at least over 1000).
>
> My configuration is what seems to be the usual thing documented say  
> in Rails Cookbook, on various blogs, and so on.  I can post the  
> details if needed.
>
> What would folks suggest, or what comments do you have?
>
>
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users

When you are testing Apache, is it configured exactly the same as when you
are using it to front Mongrel?  i.e. do you have the mod_proxy_balancer
module setup and configured, with Mongrel running, etc.?  I am doing that.
So, it''s likely I could get it faster if I didn''t have any of
that.


On 2/27/07, Kirk Haines <wyhaines at gmail.com>
wrote:
>
> On 2/27/07, Christopher Bailey <chris at codeintensity.com> wrote:
>
> > What I''m running into is that Mongrel appears only half as
fast as
> Apache
> > when serving a small static HTML file.  If I then add in Apache with
> > mod_proxy_balancer, going to a single Mongrel, it drops down to nearly
> about
> > a third of what pure static Apache will do.  This seems bogus to me,
and
> I
> > suspect I have either some configuration problem, or something.  My
> > understanding from what I''ve read is that Mongrel should be
fairly close
> to
> > Apache when serving static content (at least not only 50% as fast). 
Is
> that
> > right as a generalization?
>
> That may not be far off at all, and that difference between going
> directly to Mongrel versus going through Apache with proxying that you
> report is about what I have seen.
>
> > - Just Apache, num-conns=15000, ~1400 req/sec
>
> That actually seems really slow.  My box is a lot slower than yours
> (dual 2ghz 32 bit AMD processors on 2gb RAM, older kernel, slower
> disk), and I get twice that speed through Apache (2.0) to a small
> static file.  About 2800/second.
>
> > - Direct to Mongrel on port 5000, num-conns=8000, ~740 req/sec
>
> This is close to what I have seen in my direct-to-mongrel tests,
> though I get speeds that are a little bit higher.  In the 800+
> req/second range.  I don''t think anything is wrong with your
tests.
>
>
> Kirk Haines
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/mongrel-users/attachments/20070227/69ae50dd/attachment.html

On 2/27/07, Christopher Bailey <chris at codeintensity.com>
wrote:
> When you are testing Apache, is it configured exactly the same as when you
> are using it to front Mongrel?  i.e. do you have the mod_proxy_balancer
> module setup and configured, with Mongrel running, etc.?  I am doing that.
> So, it''s likely I could get it faster if I didn''t have
any of that.
The configuration is identical, and this server is, at any given
moment in time, handling about 65 sites/apps so there are always other
requests being handled by Apache, and many other backend processes,
some of which are mongrels, alive at the same time.  It''s always
pretty consistently delivered around the same number of requests per
second.

In any event, though, when running a (non-rails) mongrel only static
file delivery test, I get speeds a bit higher than what you reported.
ab and httperf report about the same numbers, and they are around 900,
give or take 50.  I just ran a few hundered thousand requests to pull
some solid current timings, and some 10 second bursts were as high as
1000 requests/second or as low as 800/second, but most were centered
around 900.


Kirk Haines

Moving forward, I may not understand httperf''s num-conns, and rate, as
applied to scaling up the number of Mongrels.  Either that, or my server has
issues.

So, as per Zed''s email, let''s say you establish your baseline
of what your
Apache and 1 Mongrel can do.  Let''s say that''s 450 req/sec and
since it''s
single, that''s 450 conn/sec, as a result of this command:

httperf --hog --server lab05 --port 80 --uri /mongrel_alive.html --num-conns
4500 --rate 450

This gives me back 10 seconds of time, and 450 req/sec, 450 conn/sec.

So, now I add a Mongrel into the mix, restart Apache, etc.  I would think
that I can now use:

httperf --hog --server lab05 --port 80 --uri /mongrel_alive.html --num-conns
9000 --rate 900

This doubles the rate, and doubles the number of connections, but since I
have a supposed double handling capacity, I should yield the same total time
(of 10 seconds).  Is that right?  Watching this in top I do see that both
Mongrels are grabbing CPU time.

In my case it starts breaking down with just two Mongrels (and to be clear,
I was not running top during the tests).  I drop to about 350 req/sec
without errors.


On 2/27/07, Christopher Bailey <chris at codeintensity.com>
wrote:
>
> When you are testing Apache, is it configured exactly the same as when you
> are using it to front Mongrel?  i.e. do you have the mod_proxy_balancer
> module setup and configured, with Mongrel running, etc.?  I am doing that.
> So, it''s likely I could get it faster if I didn''t have
any of that.
>
>
> On 2/27/07, Kirk Haines <wyhaines at gmail.com> wrote:
> >
> > On 2/27/07, Christopher Bailey <chris at codeintensity.com>
wrote:
> >
> > > What I''m running into is that Mongrel appears only half
as fast as
> > Apache
> > > when serving a small static HTML file.  If I then add in Apache
with
> > > mod_proxy_balancer, going to a single Mongrel, it drops down to
nearly
> > about
> > > a third of what pure static Apache will do.  This seems bogus to
me,
> > and I
> > > suspect I have either some configuration problem, or something. 
My
> > > understanding from what I''ve read is that Mongrel should
be fairly
> > close to
> > > Apache when serving static content (at least not only 50% as
> > fast).  Is that
> > > right as a generalization?
> >
> > That may not be far off at all, and that difference between going
> > directly to Mongrel versus going through Apache with proxying that you
> > report is about what I have seen.
> >
> > > - Just Apache, num-conns=15000, ~1400 req/sec
> >
> > That actually seems really slow.  My box is a lot slower than yours
> > (dual 2ghz 32 bit AMD processors on 2gb RAM, older kernel, slower
> > disk), and I get twice that speed through Apache (2.0) to a small
> > static file.  About 2800/second.
> >
> > > - Direct to Mongrel on port 5000, num-conns=8000, ~740 req/sec
> >
> > This is close to what I have seen in my direct-to-mongrel tests,
> > though I get speeds that are a little bit higher.  In the 800+
> > req/second range.  I don''t think anything is wrong with your
tests.
> >
> >
> > Kirk Haines
> > _______________________________________________
> > Mongrel-users mailing list
> > Mongrel-users at rubyforge.org
> > http://rubyforge.org/mailman/listinfo/mongrel-users
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/mongrel-users/attachments/20070227/502b2de0/attachment.html

Would you mind sharing your specific httperf commands?

On 2/27/07, Kirk Haines <wyhaines at gmail.com>
wrote:
>
> On 2/27/07, Christopher Bailey <chris at codeintensity.com> wrote:
> > When you are testing Apache, is it configured exactly the same as when
> you
> > are using it to front Mongrel?  i.e. do you have the
mod_proxy_balancer
> > module setup and configured, with Mongrel running, etc.?  I am doing
> that.
> > So, it''s likely I could get it faster if I didn''t
have any of that.
>
> The configuration is identical, and this server is, at any given
> moment in time, handling about 65 sites/apps so there are always other
> requests being handled by Apache, and many other backend processes,
> some of which are mongrels, alive at the same time.  It''s always
> pretty consistently delivered around the same number of requests per
> second.
>
> In any event, though, when running a (non-rails) mongrel only static
> file delivery test, I get speeds a bit higher than what you reported.
> ab and httperf report about the same numbers, and they are around 900,
> give or take 50.  I just ran a few hundered thousand requests to pull
> some solid current timings, and some 10 second bursts were as high as
> 1000 requests/second or as low as 800/second, but most were centered
> around 900.
>
>
> Kirk Haines
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/mongrel-users/attachments/20070227/695fed05/attachment.html

I understand that, but what I was trying to do was establish some baseline
numbers so that I could ensure I was doing my testing properly.  We actually
already have Apache serving the static stuff.  But by trying a static file
with Mongrel, I had figured this could set a baseline and I could double
check scaling up the number of mongrels to make sure my test methodology was
correct, before going to Rails and complicating things.

My next step is replacing this with hitting a Rails view that has no DB
query in it.  So, essentially a static Rails view.  In doing this, the
performance is slower of course.  Just moreso than I''d hoped for and of
course by now we''re way off from static Apache.  If I hit a single
Mongrel
instance directly (no Apache), on a tiny page (almost empty actually), I
get:
~54 req/sec
~105 req/sec with disabled sessions (not surprising given that right now
it''s set up to use the database for sessions (DB is running on same
machine
as this for the tests))
~184 req/sec with disabled sessions and disabling AuthenticatedSystem.
~415 req/sec if the page is a cached page (basically the same speed as
hitting an actual static page)

So, I guess this is mostly a good lesson for me in the performance of the
system.  The only issue I''m really facing is that adding Mongrels
doesn''t
seem to scale much, even by just bumping it up to 2.  With the above test,
in the case where I was getting about ~184 req/sec, if I double the rate and
num-conns for httperf with now running 2 Mongrels, I can''t get double
the
results, in fact, I can''t even get 300 req/sec without errors (and
yielding
208req/sec).


On 2/27/07, Christos Zisopoulos <christos at fetaphunk.com>
wrote:
>
> Christopher,
>
> Mongrel is not meant to serve static files. It is meant to replace
> the FCGI part of doing Apache->FCGI->Ruby. Or
WhateverBalancer->FCGI-
> >Ruby.
>
> As Ezra put its here, http://f8p.com/96ktde , "...it is horribly
> ineficient to serve static files with mongrel..."
>
> Most mongrel setups (ones with Apache) use mod_rewrite to server the
> static files through apache (including cached rails html files).
>
> Try this vhost template ( http://f8p.com/6d6i0s ) that uses
> mod_rewrite to serve static HTML through Apache.
>
> -christos
>
> On 28 Feb 2007, at 01:48, Christopher Bailey wrote:
>
> > As additional info, I ran the same kind of tests with Pen and Nginx
> > instead of Apache+mod_proxy_balancer, just to get an idea on that
> > aspect:
> >
> > Pen with 1 Mongrel: ~671 req/sec
> > Nginx with 1 Mongrel: ~612 req/sec
> >
> > So these at least show a smaller overhead for the load balancer
> > aspect.  They also are more bare-minimum configured (I don''t
have
> > much experience with either of those, so just did the basic Pen
> > setup where you tell it what port to run on and the ports/domain
> > +port it''s proxying).
> >
> > On 2/27/07, Christopher Bailey <chris at codeintensity.com>
wrote: I''m
> > trying to do some initial benchmarking of our setup, mainly just to
> > establish baselines.  I''m essentially using the process Zed
> > outlines in a previous message:
> > http://rubyforge.org/pipermail/mongrel-users/2006-May/000200.html
> >
> > What I''m running into is that Mongrel appears only half as
fast as
> > Apache when serving a small static HTML file.  If I then add in
> > Apache with mod_proxy_balancer, going to a single Mongrel, it drops
> > down to nearly about a third of what pure static Apache will do.
> > This seems bogus to me, and I suspect I have either some
> > configuration problem, or something.  My understanding from what
> > I''ve read is that Mongrel should be fairly close to Apache
when
> > serving static content (at least not only 50% as fast).  Is that
> > right as a generalization?
> >
> > Here''s some info to back this up.
> > - Server: HP DL360 dual 3.0GHz Xeons, 4GB RAM, 10k RPM SCSI disks
> > - OS: Fedora Core 6, up to date as of a few days ago, kernel 2.6.18
> > smp
> > - Apache 2.2.3, with mod_proxy_balancer (the RPM install)
> > - Mongrel 1.0.1, mongrel_cluster 0.2.1
> > - ruby 1.8.5 (2006-12-04 patchlevel 2)
> > - serving a static HTML file from the Rails app''s public
directory
> > - Testing using an identical server, that sits above this one in
> > the rack, connected to a switch (so it''s one hop)
> > - Using httperf for testing, with the following command:
> >   httperf --server lab05 --port 80 --uri /mongrel_alive.html --num-
> > conns 10000
> > The number of connections I vary to be near the 10 second mark...
> >
> > Here''s the results:
> >
> > - Just Apache, num-conns=15000, ~1400 req/sec
> > - Direct to Mongrel on port 5000, num-conns=8000, ~740 req/sec
> > - Apache mod_proxy_balancer to a single Mongrel, num-conns=5000,
> > ~475 req/sec
> >
> > So, I''m incurring a massive penalty for the balancer/cluster
> > setup.  In production we will be using hardware load balancers, but
> > even still, my understanding was that instead of a drop from 1400
> > to 740, it should be somewhat closer (say at least over 1000).
> >
> > My configuration is what seems to be the usual thing documented say
> > in Rails Cookbook, on various blogs, and so on.  I can post the
> > details if needed.
> >
> > What would folks suggest, or what comments do you have?
> >
> >
> > _______________________________________________
> > Mongrel-users mailing list
> > Mongrel-users at rubyforge.org
> > http://rubyforge.org/mailman/listinfo/mongrel-users
>
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/mongrel-users/attachments/20070228/27df05c8/attachment-0001.html

On Tue, 27 Feb 2007 15:40:30 -0800
"Christopher Bailey" <chris at codeintensity.com> wrote:
> I''m trying to do some initial benchmarking of our setup, mainly
just to
> establish baselines.  I''m essentially using the process Zed
outlines in a
> previous message:
> http://rubyforge.org/pipermail/mongrel-users/2006-May/000200.html
Don''t forget, this is also available which is pretty much the same
document/process:

http://mongrel.rubyforge.org/docs/how_many_mongrels.html
> - Using httperf for testing, with the following command:
>   httperf --server lab05 --port 80 --uri /mongrel_alive.html --num-conns
> 10000
That''s fine for a httperf command.
> Here''s the results:
> 
> - Just Apache, num-conns=15000, ~1400 req/sec
> - Direct to Mongrel on port 5000, num-conns=8000, ~740 req/sec
> - Apache mod_proxy_balancer to a single Mongrel, num-conns=5000, ~475
> req/sec
The apache numbers look way off.
> So, I''m incurring a massive penalty for the balancer/cluster
setup.  In
> production we will be using hardware load balancers, but even still, my
> understanding was that instead of a drop from 1400 to 740, it should be
> somewhat closer (say at least over 1000).
Well, it''s not "massive" but you should test other
configurations and
validate that you have apache configured right.  My first thought is
that your apache configuration is slow to begin with, which might mean
that it''s not tuned properly.  There''s several different
workers you
can use so try them out and see which is the best.  Also, try nginx and
your hardware load balancer.

Your next thing to do is see if you can get a very small sample RoR
application that just renders "test" to perform.  If RoR performs less
than 475/sec then there''s no point in stressing over the drop in
performance because of apache.  You''ll just have to know that you
won''t
be able to push it above that.

Finally, you may be worrying about this way too much in the beginning.
You''ve measured there''s a potential problem, you should then
try to
rule it out (apache is the hypothetical cause), but if you can''t then
this just factors into your planning for the production deployment.
It''s not the end of the world.

-- 
Zed A. Shaw, MUDCRAP-CE Master Black Belt Sifu
http://www.zedshaw.com/
http://www.awprofessional.com/title/0321483502 -- The Mongrel Book
http://mongrel.rubyforge.org/
http://www.lingr.com/room/3yXhqKbfPy8 -- Come get help.

On Tue, 27 Feb 2007 22:02:58 -0800
"Christopher Bailey" <chris at codeintensity.com> wrote:
> Moving forward, I may not understand httperf''s num-conns, and
rate, as
> applied to scaling up the number of Mongrels.  Either that, or my server
has
> issues.
> 
> httperf --hog --server lab05 --port 80 --uri /mongrel_alive.html
--num-conns
> 4500 --rate 450
> 
> This gives me back 10 seconds of time, and 450 req/sec, 450 conn/sec.
Try inching that up incrementally until you find the "sweet spot".
It''s probably going to be 450 or near it, but try 500.  You''ll
know you
passed the sweet spot when the req/sec crashes suddenly.  I do kind of
a binary search algo until I find the right number.
> So, now I add a Mongrel into the mix, restart Apache, etc.  I would think
> that I can now use:
> 
> httperf --hog --server lab05 --port 80 --uri /mongrel_alive.html
--num-conns
> 9000 --rate 900
You won''t get 900 right away, especially since you have one CPU and are
maxing them out but if you get 350 req/sec then try again inching it
down until you get a steady number.  Try 800 to see if that gives you
800 req/sec, then further down until you get it, then inch up to see
where the sweet spot is.
> In my case it starts breaking down with just two Mongrels (and to be clear,
> I was not running top during the tests).  I drop to about 350 req/sec
> without errors.
Yep, and that''s just kind of the way it goes.  In your case, having
more mongrels isn''t going to get you higher performance, so just add
one more mongrel and use your mongrel processes more to ensure
concurrency when running Rails.  Also, you haven''t even hit testing
your rails performance yet, do that too and just kind of accept it as
your speed then plan for it.

-- 
Zed A. Shaw, MUDCRAP-CE Master Black Belt Sifu
http://www.zedshaw.com/
http://www.awprofessional.com/title/0321483502 -- The Mongrel Book
http://mongrel.rubyforge.org/
http://www.lingr.com/room/3yXhqKbfPy8 -- Come get help.

Ok, thanks.  Yes, it seems that basically things are somewhat as expected.
It''s definitely a good exercise in knowing how noticeably certain
things
(like sessions, etc.) affect the performance.  At this point I''ve
removed
Apache from the mix, as we wouldn''t be using it anyway (we use it in
dev as
it''s what we have now that allows us to easily run static, and multiple
Rails on a single box and balance across that, etc.  But, I will probably
look at Nginx as an alternative.  For my tests I''ve now switched to Pen
as
that seems the least obtrusive, and is about as simple as we could get
before having no software load balance/closes to hardware LB.

On 2/28/07, Zed A. Shaw <zedshaw at zedshaw.com>
wrote:
>
> On Tue, 27 Feb 2007 22:02:58 -0800
> "Christopher Bailey" <chris at codeintensity.com> wrote:
>
> > Moving forward, I may not understand httperf''s num-conns, and
rate, as
> > applied to scaling up the number of Mongrels.  Either that, or my
server
> has
> > issues.
> >
> > httperf --hog --server lab05 --port 80 --uri /mongrel_alive.html
> --num-conns
> > 4500 --rate 450
> >
> > This gives me back 10 seconds of time, and 450 req/sec, 450 conn/sec.
>
> Try inching that up incrementally until you find the "sweet
spot".
> It''s probably going to be 450 or near it, but try 500. 
You''ll know you
> passed the sweet spot when the req/sec crashes suddenly.  I do kind of
> a binary search algo until I find the right number.
>
> > So, now I add a Mongrel into the mix, restart Apache, etc.  I would
> think
> > that I can now use:
> >
> > httperf --hog --server lab05 --port 80 --uri /mongrel_alive.html
> --num-conns
> > 9000 --rate 900
>
> You won''t get 900 right away, especially since you have one CPU
and are
> maxing them out but if you get 350 req/sec then try again inching it
> down until you get a steady number.  Try 800 to see if that gives you
> 800 req/sec, then further down until you get it, then inch up to see
> where the sweet spot is.
>
> > In my case it starts breaking down with just two Mongrels (and to be
> clear,
> > I was not running top during the tests).  I drop to about 350 req/sec
> > without errors.
>
> Yep, and that''s just kind of the way it goes.  In your case,
having
> more mongrels isn''t going to get you higher performance, so just
add
> one more mongrel and use your mongrel processes more to ensure
> concurrency when running Rails.  Also, you haven''t even hit
testing
> your rails performance yet, do that too and just kind of accept it as
> your speed then plan for it.
>
> --
> Zed A. Shaw, MUDCRAP-CE Master Black Belt Sifu
> http://www.zedshaw.com/
> http://www.awprofessional.com/title/0321483502 -- The Mongrel Book
> http://mongrel.rubyforge.org/
> http://www.lingr.com/room/3yXhqKbfPy8 -- Come get help.
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/mongrel-users/attachments/20070228/5535dcac/attachment.html