Backgroundrb devel - Jan 2009 - Configure multiple workers for persistent job queue‏

Hi,

I''m using a persistent job queue with a single worker thread
to process the jobs.  However the jobs are being created faster than
the worker can handle them.  Is it possible to configure backgroundrb
to start multiple copies of the same worker to pick up these jobs?

Thanks,
Mark
_________________________________________________________________
Send e-mail faster without improving your typing skills.
http://windowslive.com/online/hotmail?ocid=TXT_TAGLM_WL_hotmail_acq_speed_122008
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20090106/eedd6184/attachment.html>

2009/1/7 Mark D <buenos79 at hotmail.com>:
> Hi,
>
> I''m using a persistent job queue with a single worker thread to
process the
> jobs.  However the jobs are being created faster than the worker can handle
> them.  Is it possible to configure backgroundrb to start multiple copies of
> the same worker to pick up these jobs?
>
1. You can use one worker as master (the one that takes jobs out of
the queue) and have that worker start multiple copies of itself (using
MiddleMan.new_worker) and hand over the jobs to these workers.

2. There isn''t a builtin functionality of auto-adjusting worker count
if number of tasks in the queue is high. But if your tasks are
inherently IO bound, you can use remove item from queue and run it
inside thread pool.