On May 29, 2006, at 4:27 AM, Hej p? er wrote:
> Running rails, lighttpd and scgi.
> If I use only one listener is it right that the server will only be
> able
> to serve one request at a time? I have one action that takes a long
> time
> to complete (makes a system call to a java app), when this action is
> being called the whole server seems to be locked, however, all other
> requests (from other clients) that come in during this period are
> being
> qeued up and served as soon as the lengthy action is finished. Is it
> possible to serve all other requests as soon as they come in? I
> tried to
> put the system call in a thread, but that didnt help.
>
> Will multiple listeners solve the performance problem?
> How many listeners can one have?
>
> --
Hello-
If you are trying to run long running background tasks then you can
do a couple of different things. It is true that while your long task
is running it will block the fcgi listener it is running in. So you
need to have more then one listener so things don''t block. But this
doesn''t scale that well because even if you have 5 listeners, what if
you get 10 users trying to run the long action all at once? 5 will
run and the rest will just queue up and wait until they get a free
slot to run in. Fcgi processes consume anywhere between 20Mb-80Mb
depending on what you are doing in your application. So adding more
of these just to be able to run your long running tasks is not very
practical. Especially because just a few, maybe two or three fcgi''s
or mongrels can serve a *lot* of traffic as long as the requests
don''t take a really long time to run.
So the better way to handle the long running tasks IMHO is to
separate them from the rails request/response cycle so they don''t tie
up your listeners. I wrote BackgrounDRb Plugin[1] just for this exact
situation. It works by creating a DRb(distributed ruby) object server
that you can run your long tasks in. You kick off a worker class from
within rails but it doesn''t block because it gets run in the
BackgrounDRb server. Then you can query your tasks while its running
via ajax so you can update a progress bar in the browser while your
task runs.
Anyway, you should check out BackgrounDRb and see if you can work it
into your setup. This way you could get away with just two or three
fcgi listerners for all your traffic and run your long tasks in the
drb server. A BackgrounDRb server only uses 2Mb of ram when it is
idle so it is much more resource friendly then trying to make 20
listeners or something.
Cheers-
-Ezra
[1] http://brainspl.at/articles/2006/05/25/backgroundrb-new-release