Eventmachine talk - Feb 2007 - EM based clustering proxy for web apps

Just figured I''d send out a preliminary note letting you all know that
I am working on a clustering proxy for web apps that uses EM.

It was born out of a need for a simple clustering deployment solution
for IOWA that carries with it a little bit more intelligence than the
typical load balancing proxy does.

I am building it with features that I want for it to work with IOWA,
but intend it to be usable with other web frameworks, as well.

I should be doing an initial release within about a week, but my
initial testing is pretty satisfying.

The current state of it will, on my test platform (an older dual
processor box running 2ghz AMD athlon 32-bit cpus and a linux 2.4
kernel), running to a pair of IOWA backends, deliver static files at
the same speed that Apache does on this system (2800/second).  And
testing using a copy of a real, production site show it to be very
effective so far.

A site that dynamically renders about 200 pages per second through a
single IOWA process will render 360/second or so through the
clustering proxy to two backends, and a site that will do 90/second,
with a database hit happening on every request (db server on the same
box), will reliably do 160/second with two backends there, as well.


Kirk Haines

On 2/25/07, Kirk Haines <wyhaines at gmail.com>
wrote:
> Just figured I''d send out a preliminary note letting you all know
that
> I am working on a clustering proxy for web apps that uses EM.
>
> It was born out of a need for a simple clustering deployment solution
> for IOWA that carries with it a little bit more intelligence than the
> typical load balancing proxy does.
>
> I am building it with features that I want for it to work with IOWA,
> but intend it to be usable with other web frameworks, as well.
>
> I should be doing an initial release within about a week, but my
> initial testing is pretty satisfying.
>
> The current state of it will, on my test platform (an older dual
> processor box running 2ghz AMD athlon 32-bit cpus and a linux 2.4
> kernel), running to a pair of IOWA backends, deliver static files at
> the same speed that Apache does on this system (2800/second).  And
> testing using a copy of a real, production site show it to be very
> effective so far.
>
> A site that dynamically renders about 200 pages per second through a
> single IOWA process will render 360/second or so through the
> clustering proxy to two backends, and a site that will do 90/second,
> with a database hit happening on every request (db server on the same
> box), will reliably do 160/second with two backends there, as well.
Great work, Kirk, I was thinking myself of doing something like that.
Since Apache2.2''s default load balancer isn''t quite ideal for
rails,
because it can load same mongrels over over again.

I thought there should be a proxy, which detects before forwarding a
request to one of the mongrels, if that particular instance of mongrel
is busy or something.


--
gnufied
-----------
There was only one Road; that it was like a great river: its springs
were at every doorstep, and every path was its tributary.
http://people.inxsasia.com/~hemant

On 2/24/07, Kirk Haines <wyhaines at gmail.com>
wrote:
>
> Just figured I''d send out a preliminary note letting you all know
that
> I am working on a clustering proxy for web apps that uses EM.


Very encouraging, Kirk. Is anyone out there interested on working on getting
Kirk''s cluster-proxy to run with Rails?
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/eventmachine-talk/attachments/20070226/12c28947/attachment.html

On 2/26/07, Francis Cianfrocca <garbagecat10 at gmail.com>
wrote:
>
> Very encouraging, Kirk. Is anyone out there interested on working on
getting
> Kirk''s cluster-proxy to run with Rails?
My goal is to release a first cut by the end of the week.  Then I will
look at what is necessary to alter the Rails Mongrel handler and the
Nitro Mongrel handlers to create a clustering version, after I''ve
worked out some more details through my experimentation with IOWA, and
I may need some assistance with that, especially with testing and
tuning.


Kirk

Hey how could I get the IP number of the remote connection... I forgot...
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/eventmachine-talk/attachments/20070226/7946199c/attachment.html

On 2/26/07, Kirk Haines <wyhaines at gmail.com>
wrote:
>
> On 2/26/07, Francis Cianfrocca <garbagecat10 at gmail.com> wrote:
> >
> > Very encouraging, Kirk. Is anyone out there interested on working on
> getting
> > Kirk''s cluster-proxy to run with Rails?
>
> My goal is to release a first cut by the end of the week.  Then I will
> look at what is necessary to alter the Rails Mongrel handler and the
> Nitro Mongrel handlers to create a clustering version, after I''ve
> worked out some more details through my experimentation with IOWA, and
> I may need some assistance with that, especially with testing and
> tuning.


 Looking forward to that. The Rails dispatch code is pretty small and would
not be hard to adapt to this purpose. This project might be a good way to
give Rails users some options.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/eventmachine-talk/attachments/20070226/99f3676e/attachment.html

Connection#get_peername

On 2/26/07, Daniel Aquino <mr.danielaquino at gmail.com>
wrote:
>
> Hey how could I get the IP number of the remote connection... I forgot...
>
> _______________________________________________
> Eventmachine-talk mailing list
> Eventmachine-talk at rubyforge.org
> http://rubyforge.org/mailman/listinfo/eventmachine-talk
>


--
Tony Arcieri
ClickCaster, Inc.
tony at clickcaster.com
(970) 232-4208
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/eventmachine-talk/attachments/20070226/6e9dd775/attachment-0001.html

On 2/26/07, Tony Arcieri <tony at clickcaster.com>
wrote:
>
> Connection#get_peername


Tony gave the right answer but if I remember correctly, there are some
connection types that Connection#get_peername might  not work with. Like
Unix-domain sockets. It''ll definitely work with TCP, though. Patches
gratefully accepted.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/eventmachine-talk/attachments/20070226/eced8bd8/attachment.html

Oh, by the way, as a companion to the clustering proxy I''m also
putting together a simple asynchronous logging service using EM.
Setting it up to so that logging messages can be dumped to it very
quickly, and it''ll take care of getting the messages written to their
log file(s), clustering writes as necessary if messages are coming in
quickly.

The intention is that this should allow a collection of cluster
processes all running the same thing to have their messages written to
a single log, without incurring any IO locking bottlenecks from having
multiple processes all trying to send data to the same file.


Kirk Haines

On 2/27/07, Kirk Haines <wyhaines at gmail.com>
wrote:
> Oh, by the way, as a companion to the clustering proxy I''m also
> putting together a simple asynchronous logging service using EM.
> Setting it up to so that logging messages can be dumped to it very
> quickly, and it''ll take care of getting the messages written to
their
> log file(s), clustering writes as necessary if messages are coming in
> quickly.
>
> The intention is that this should allow a collection of cluster
> processes all running the same thing to have their messages written to
> a single log, without incurring any IO locking bottlenecks from having
> multiple processes all trying to send data to the same file.
What are the advantages over syslogd(8)?

--Michael

On 2/27/07, Michael S. Fischer <michael at dynamine.net> wrote:
> What are the advantages over syslogd(8)?
I see a few.

People can use it without having root access on a system.  They just
need to be able to run persistent processes.

syslog uses UDP, so logging messages can be lost, especially if there
are lots of packets flying around.  This one is a big one to me.

syslog is only an option on *nix systems.


Kirk Haines

> On 2/27/07, Michael S. Fischer <michael at dynamine.net> wrote:
>
> > What are the advantages over syslogd(8)?
Oh, one more.  syslog is very limited with regard to it''s facility
support.

There is local0 through local7, but that''s not a lot of flexibility
if, as is true in my most common hosting scenario, I have a large
number of sites I want to log for.


Kirk Haines

On 2/27/07, Kirk Haines <wyhaines at gmail.com>
wrote:
> On 2/27/07, Michael S. Fischer <michael at dynamine.net> wrote:
>
> > What are the advantages over syslogd(8)?
>
> I see a few.
>
> People can use it without having root access on a system.  They just
> need to be able to run persistent processes.
>
> syslog uses UDP, so logging messages can be lost, especially if there
> are lots of packets flying around.  This one is a big one to me.
See also syslog-ng at http://www.balabit.com/products/syslog_ng/

--Michael

On 2/27/07, Kirk Haines <wyhaines at gmail.com>
wrote:
>
> Oh, by the way, as a companion to the clustering proxy I''m also
> putting together a simple asynchronous logging service using EM.
> Setting it up to so that logging messages can be dumped to it very
> quickly, and it''ll take care of getting the messages written to
their
> log file(s), clustering writes as necessary if messages are coming in
> quickly.
>
> The intention is that this should allow a collection of cluster
> processes all running the same thing to have their messages written to
> a single log, without incurring any IO locking bottlenecks from having
> multiple processes all trying to send data to the same file.


How are you planning to do the comms to the logging engine? Unix-domain
socket? Localhost TCP? I agree that UDP is a sub-ideal choice when you
really need to be sure. Another thing I''ve done is have a
logging-facility
automatically write into a ferret index. Gives full-text searchable log
files.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/eventmachine-talk/attachments/20070227/bcc7fe6a/attachment.html

On 2/27/07, Francis Cianfrocca <garbagecat10 at gmail.com> wrote:
> How are you planning to do the comms to the logging engine? Unix-domain
> socket? Localhost TCP? I agree that UDP is a sub-ideal choice when you
> really need to be sure. Another thing I''ve done is have a
logging-facility
> automatically write into a ferret index. Gives full-text searchable log
> files.
A configurable TCP port.  It''s not really a whole lot different than
the clustering proxy code along with the backend protocol that keeps
the connection alive.


Kirk

On 2/27/07, Kirk Haines <wyhaines at gmail.com> wrote:
> A configurable TCP port.  It''s not really a whole lot different
than
> the clustering proxy code along with the backend protocol that keeps
> the connection alive.
It''s stupid fast.  Wow.  My initial naive tests have it taking in
excess of 55000 messages per second on this box.


Kirk Haines