Backgroundrb devel - Feb 2008 - worker process hanging around, spiking memory

Hi. Posted a general question earlier today.  I have a specific
question this time -- hoping someone can help out.  I think I''m
probably doing something incorrectly.  Here goes:

I have a Worker method that sends a lot of emails.  The method looks like this:


def send_participant_reminders(args=nil)
    args[:emails].each do |email|

      unless email.notified?
        Emailer.deliver_survey_reminder(email)
      end

    end
 end


I call that worker from a controller like so:

    MiddleMan.new_worker(:worker => :participant_worker)
    MiddleMan.ask_work(:worker => :participant_worker,
      :worker_method => :send_participant_reminders,
      :data => {:emails => @emails})

----------------------------

My emails are being sent, but I''m seeing a new process created *each
time I invoke the code in the controller*.  That process starts out at
about 17m of memory then, once all work appears to have been
completed, jumps to about 50m.  The process stays around until
"script/backgroundrb stop".

Maybe I''ve been reading the examples incorrectly?

Any help would be appreciated.  Seeing this same problem (I think)
suck up 4G of ram on a production server.  yuck.

Thanks again,
clint

Hi Clint,

On Wed, Feb 27, 2008 at 3:26 PM, Clint Troxel <clint at ctro.net> wrote:
> Hi. Posted a general question earlier today.  I have a specific
> question this time -- hoping someone can help out.  I think I''m
> probably doing something incorrectly.  Here goes:
>
> I have a Worker method that sends a lot of emails.  The method looks like
> this:
>
>
> def send_participant_reminders(args=nil)
>    args[:emails].each do |email|
>
>      unless email.notified?
>        Emailer.deliver_survey_reminder(email)
>      end
>
>    end
>  end
>
>
> I call that worker from a controller like so:
>
>    MiddleMan.new_worker(:worker => :participant_worker)
>    MiddleMan.ask_work(:worker => :participant_worker,
>      :worker_method => :send_participant_reminders,
>      :data => {:emails => @emails})
>
> ----------------------------
>
> My emails are being sent, but I''m seeing a new process created
*each
> time I invoke the code in the controller*.  That process starts out at
> about 17m of memory then, once all work appears to have been
> completed, jumps to about 50m.  The process stays around until
> "script/backgroundrb stop".
>
> Maybe I''ve been reading the examples incorrectly?
>
> Any help would be appreciated.  Seeing this same problem (I think)
> suck up 4G of ram on a production server.  yuck.
>
> Thanks again,
> clint

Calling new_worker does indeed create a new backgroundrb process (which may
or may not be necessary in your case, see below).  If you do in fact need to
create a new process for each task, you''ll need to call either exit
(from
within the worker) or MiddleMan.delete_worker (from the controller) once the
task is done. If you need to keep track of these worker tasks while
they''re
running (to call delete_worker from the controller, or to use ask_status),
you should also use a job_key when you create it.

You may not actually need the new process, however.  If the likely use case
is that there will only be one of these tasks running at a time, then you
can just omit the new_worker call entirely and direct all calls to a single
running instance of the worker process.  You may also want to investigate
using thread_pool.defer inside the worker to run the individual tasks in
threads (rather than spawning new worker processes for each one) if it seems
likely that there will be multiple requests running simultaneously.

Thanks,
Greg
p.s. If I''ve misrepresented anything, I''d like to invite
Hemant or anyone
else more knowledgeable than I am to correct me, as I''ve only started
working on backgroundrb in the last few weeks.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080227/aa903a9b/attachment.html

Greg,

that is helpful.  thanks!

calling exit() from the workers does indeed kill the process.

Two more questions:

1) before the process is killed by exit() I repeatedly see the memory
of the bdrb process spike from 18 to 45 M -- is there an explanation
for this?  Also, the process seems to take a long time to finish --
minutes just to send 4 emails.  Could some sort of configuration be
causing behavior like this?

2) More importantly, what is the recommended version to run?  And, how
do I check the version in my vendor dir? (I can''t remember what
version you were at when I imported into my rails proj)

Thanks again,
-clint

On 2/27/08, Greg Campbell <gtcampbell at gmail.com>
wrote:
> Hi Clint,
>
>
> On Wed, Feb 27, 2008 at 3:26 PM, Clint Troxel <clint at ctro.net>
wrote:
> > Hi. Posted a general question earlier today.  I have a specific
> > question this time -- hoping someone can help out.  I think
I''m
> > probably doing something incorrectly.  Here goes:
> >
> > I have a Worker method that sends a lot of emails.  The method looks
like
> this:
> >
> >
> > def send_participant_reminders(args=nil)
> >    args[:emails].each do |email|
> >
> >      unless email.notified?
> >        Emailer.deliver_survey_reminder(email)
> >      end
> >
> >    end
> >  end
> >
> >
> > I call that worker from a controller like so:
> >
> >    MiddleMan.new_worker(:worker => :participant_worker)
> >    MiddleMan.ask_work(:worker => :participant_worker,
> >      :worker_method => :send_participant_reminders,
> >      :data => {:emails => @emails})
> >
> > ----------------------------
> >
> > My emails are being sent, but I''m seeing a new process
created *each
> > time I invoke the code in the controller*.  That process starts out at
> > about 17m of memory then, once all work appears to have been
> > completed, jumps to about 50m.  The process stays around until
> > "script/backgroundrb stop".
> >
> > Maybe I''ve been reading the examples incorrectly?
> >
> > Any help would be appreciated.  Seeing this same problem (I think)
> > suck up 4G of ram on a production server.  yuck.
> >
> > Thanks again,
> > clint
>
> Calling new_worker does indeed create a new backgroundrb process (which may
> or may not be necessary in your case, see below).  If you do in fact need
to
> create a new process for each task, you''ll need to call either
exit (from
> within the worker) or MiddleMan.delete_worker (from the controller) once
the
> task is done. If you need to keep track of these worker tasks while
they''re
> running (to call delete_worker from the controller, or to use ask_status),
> you should also use a job_key when you create it.
>
> You may not actually need the new process, however.  If the likely use case
> is that there will only be one of these tasks running at a time, then you
> can just omit the new_worker call entirely and direct all calls to a single
> running instance of the worker process.  You may also want to investigate
> using thread_pool.defer inside the worker to run the individual tasks in
> threads (rather than spawning new worker processes for each one) if it
seems
> likely that there will be multiple requests running simultaneously.
>
> Thanks,
> Greg
> p.s. If I''ve misrepresented anything, I''d like to invite
Hemant or anyone
> else more knowledgeable than I am to correct me, as I''ve only
started
> working on backgroundrb in the last few weeks.
>

On Thu, Feb 28, 2008 at 5:54 AM, Clint Troxel <clint at ctro.net>
wrote:
> Greg,
>
>  that is helpful.  thanks!
>
>  calling exit() from the workers does indeed kill the process.
>
>  Two more questions:
>
>  1) before the process is killed by exit() I repeatedly see the memory
>  of the bdrb process spike from 18 to 45 M -- is there an explanation
>  for this?  Also, the process seems to take a long time to finish --
>  minutes just to send 4 emails.  Could some sort of configuration be
>  causing behavior like this?
I do not think so. But in each process full rails environment is
loaded and hence its costly.
Thats why, you should probably just use thread_pools rather than
starting new processes.
>
>  2) More importantly, what is the recommended version to run?  And, how
>  do I check the version in my vendor dir? (I can''t remember what
>  version you were at when I imported into my rails proj)
>
Recommended version to run is one from git mainline repo:

http://gitorious.org/projects/backgroundrb/repos/mainline

We are getting ready for release 1.0.3 and I have put together new
documentation at:

http://backgroundrb.gnufied.org

Please note that above location is just temporary and in a day or two
when 1.0.3 comes out, main documentation at rubyforge will reflect the
same doco.
>
>
>  On 2/27/08, Greg Campbell <gtcampbell at gmail.com> wrote:
>  > Hi Clint,
>  >
>  >
>  > On Wed, Feb 27, 2008 at 3:26 PM, Clint Troxel <clint at
ctro.net> wrote:
>  > > Hi. Posted a general question earlier today.  I have a specific
>  > > question this time -- hoping someone can help out.  I think
I''m
>  > > probably doing something incorrectly.  Here goes:
>  > >
>  > > I have a Worker method that sends a lot of emails.  The method
looks like
>  > this:
>  > >
>  > >
>  > > def send_participant_reminders(args=nil)
>  > >    args[:emails].each do |email|
>  > >
>  > >      unless email.notified?
>  > >        Emailer.deliver_survey_reminder(email)
>  > >      end
>  > >
>  > >    end
>  > >  end
>  > >
>  > >
>  > > I call that worker from a controller like so:
>  > >
>  > >    MiddleMan.new_worker(:worker => :participant_worker)
>  > >    MiddleMan.ask_work(:worker => :participant_worker,
>  > >      :worker_method => :send_participant_reminders,
>  > >      :data => {:emails => @emails})
>  > >
>  > > ----------------------------
>  > >
>  > > My emails are being sent, but I''m seeing a new process
created *each
>  > > time I invoke the code in the controller*.  That process starts
out at
>  > > about 17m of memory then, once all work appears to have been
>  > > completed, jumps to about 50m.  The process stays around until
>  > > "script/backgroundrb stop".
>  > >
>  > > Maybe I''ve been reading the examples incorrectly?
>  > >
>  > > Any help would be appreciated.  Seeing this same problem (I
think)
>  > > suck up 4G of ram on a production server.  yuck.
>  > >
>  > > Thanks again,
>  > > clint
>  >
>  > Calling new_worker does indeed create a new backgroundrb process
(which may
>  > or may not be necessary in your case, see below).  If you do in fact
need to
>  > create a new process for each task, you''ll need to call
either exit (from
>  > within the worker) or MiddleMan.delete_worker (from the controller)
once the
>  > task is done. If you need to keep track of these worker tasks while
they''re
>  > running (to call delete_worker from the controller, or to use
ask_status),
>  > you should also use a job_key when you create it.
>  >
>  > You may not actually need the new process, however.  If the likely
use case
>  > is that there will only be one of these tasks running at a time, then
you
>  > can just omit the new_worker call entirely and direct all calls to a
single
>  > running instance of the worker process.  You may also want to
investigate
>  > using thread_pool.defer inside the worker to run the individual tasks
in
>  > threads (rather than spawning new worker processes for each one) if
it seems
>  > likely that there will be multiple requests running simultaneously.
>  >
>  > Thanks,
>  > Greg
>  > p.s. If I''ve misrepresented anything, I''d like to
invite Hemant or anyone
>  > else more knowledgeable than I am to correct me, as I''ve
only started
>  > working on backgroundrb in the last few weeks.
>  >
You have been absolutely correct!