Backgroundrb devel - Oct 2006 - BackgrounDRb 0.2.0 Release! Complete rewrite.

Hello Folks-

	I am really happy to announce the release of BackgrounDRb 0.2.0 .
This is a complete rewrite and re-architecture of this codebase. I  
want to
thank skaar for all of his hard work. I put out a call for help with  
the new
  architecture and skaar stepped up bigtime. I had already written  
the new
scheduler and cron compatible syntax and moved to multi process
architecture. But he took what I had that wasn''t very whole and did a
whopping 100 svn commits to the project! Big ups to skaar for this  
release!


= BackgrounDRb 0.2.0


!! Note that this is in an entirely different svn repo then the old  
plugin.
Make sure you remove the old before you install the new.

http://svn.devjavu.com/backgroundrb/tags/release-0.2.0/README
http://svn.devjavu.com/backgroundrb/tags/release-0.2.0 (latest release)
http://svn.devjavu.com/backgroundrb/trunk (svn trunk)
http://backgroundrb.devjavu.com (trac)
http://backgroundrb.rubyforge.org (rdoc)

Copyright (c) 2006 Ezra Zygmuntowicz and skaar[at]waste[dot]org

== DISCLAIMER

The 0.2.0 release of BackgrounDRb is a complete re-write of previous
releases and is to be considered experimental, in-complete and in many
respect untested. Our goal is to reach a release recommended for
production use by 0.3.x. This said, this release is a more robust
solution that the previous release.

Also note that support for Windows is deprecated with this release of
BackgrounDRb, sorry : ( . It may work under cygwin but we have not  
tried it.

== Introduction

BackgrounDRb is a ruby job server and scheduler. It main intent is to be
used with Ruby on Rails applications for offloading long running tasks.
Since a rails application blocks while servicing a request it is best
to move long running tasks off into a background process that is  
divorced
from the http request/response cycle.

This new release of BackgrounDRb is also modular and can be used without
Rails. So any ruby program or framework can use it.

== Technology Overview

This 0.2.x branch of BackgrounDRb introduces a completely new  
architecture
then the previous versions. Instead of a single process, multi  
threaded environment,
the new system uses multi process with IPC to manage workers. So each  
of your
workers will run in their own ruby interpreter. The interface that you
use within rails remains mostly unchanged.

The new architecture allows for a much more robust system. Each of your
worker classes get run in their own ruby interpreter. The main server  
process
holds a collection of references to your running worker objects. This  
acts as
the interface or MiddleMan that you use to interact with your remote  
jobs from
rails or other ruby client code. When workers are spawned an  
interprocess
communication channel is opened between the MiddleMan and the worker.
There is a heartbeat setup between parent and child processes so that if
you kill the parent, all children processes are killed off the next  
time they try to
hearbeat. All of this happens transparently to you as a user.

The new release comes with a brand new job scheduler. You can use
simple triggers like repeat_every or you can get more complex and use
full cron compatible syntax.

Workers can be scheduled by two built in ''trigger'' types. A
simple
''trigger'' is specified with start, stop and interval:

   require ''active_support''
   MiddleMan.schedule_worker(
     :class => :example_worker,
     :job_key => :schedule_test,
     :worker_method => :other_method,
     :trigger_args => {
       :start => Time.now + 5.seconds,
       :end => Time.now + 10.minutes,
       :repeat_interval => 30.seconds
     }

The cron trigger uses a similar syntax to cron found on UNIX systems:

   MiddleMan.schedule_worker(
     :class => simple_class,
     :job_key => :schedule_test,
     :worker_method => :arg_method,
     :worker_method_args => "my argument to arg_method",
     :trigger_args => "0 15 10 * * * *"
   )


Also note that when the server starts up, you will see 3 processes  
running.
One of the is the MiddleMan server, one is the results worker and one is
the logger worker. When you do a logger.info("foo log!") in your  
workers,
you are actually logging to the Logger worker.

As you might imagine, this new way of managing multiple processes will
scale a lot better then the multi threaded single process version ;) But
also be aware that there is still a thread pool in the middleman  
that  you
can control the size of. All this does is keep the plugin from  
spawning too
many processes. It will allow however many workers you specify to run at
once and any more then that will just queue up and wit for their turn to
spawn.

If you are already a user of the old BackgrounDRb please give the new
version a tryout. It should not require very much work to port your  
worker
classes to the new architecture.

This is alpha software folks. It works for me? but we will not be  
held responsible
if it ruins your day ;)

-- Ezra Zygmuntowicz 
-- Lead Rails Evangelist
-- ez at engineyard.com
-- Engine Yard, Serious Rails Hosting
-- (866) 518-YARD (9273)

First of all: Thanks you two guys out there!
It looks very nice, although it seems i wont use the new version  
since the one you gave me (pre-pre-pre version with worker pool and  
threads) works perfectly for my current project. but i will surely  
try it out.

great work!

PS: wheres the restart rake task gone? :)

Am 30.10.2006 um 02:40 schrieb Ezra Zygmuntowicz:
> Hello Folks-
>
> 	I am really happy to announce the release of BackgrounDRb 0.2.0 .
> This is a complete rewrite and re-architecture of this codebase. I
> want to
> thank skaar for all of his hard work. I put out a call for help with
> the new
>   architecture and skaar stepped up bigtime. I had already written
> the new
> scheduler and cron compatible syntax and moved to multi process
> architecture. But he took what I had that wasn''t very whole and
did a
> whopping 100 svn commits to the project! Big ups to skaar for this
> release!
>
>
> = BackgrounDRb 0.2.0
>
>
> !! Note that this is in an entirely different svn repo then the old
> plugin.
> Make sure you remove the old before you install the new.
>
> http://svn.devjavu.com/backgroundrb/tags/release-0.2.0/README
> http://svn.devjavu.com/backgroundrb/tags/release-0.2.0 (latest  
> release)
> http://svn.devjavu.com/backgroundrb/trunk (svn trunk)
> http://backgroundrb.devjavu.com (trac)
> http://backgroundrb.rubyforge.org (rdoc)
>
> Copyright (c) 2006 Ezra Zygmuntowicz and skaar[at]waste[dot]org
>
> == DISCLAIMER
>
> The 0.2.0 release of BackgrounDRb is a complete re-write of previous
> releases and is to be considered experimental, in-complete and in many
> respect untested. Our goal is to reach a release recommended for
> production use by 0.3.x. This said, this release is a more robust
> solution that the previous release.
>
> Also note that support for Windows is deprecated with this release of
> BackgrounDRb, sorry : ( . It may work under cygwin but we have not
> tried it.
>
> == Introduction
>
> BackgrounDRb is a ruby job server and scheduler. It main intent is  
> to be
> used with Ruby on Rails applications for offloading long running  
> tasks.
> Since a rails application blocks while servicing a request it is best
> to move long running tasks off into a background process that is
> divorced
> from the http request/response cycle.
>
> This new release of BackgrounDRb is also modular and can be used  
> without
> Rails. So any ruby program or framework can use it.
>
> == Technology Overview
>
> This 0.2.x branch of BackgrounDRb introduces a completely new
> architecture
> then the previous versions. Instead of a single process, multi
> threaded environment,
> the new system uses multi process with IPC to manage workers. So each
> of your
> workers will run in their own ruby interpreter. The interface that you
> use within rails remains mostly unchanged.
>
> The new architecture allows for a much more robust system. Each of  
> your
> worker classes get run in their own ruby interpreter. The main server
> process
> holds a collection of references to your running worker objects. This
> acts as
> the interface or MiddleMan that you use to interact with your remote
> jobs from
> rails or other ruby client code. When workers are spawned an
> interprocess
> communication channel is opened between the MiddleMan and the worker.
> There is a heartbeat setup between parent and child processes so  
> that if
> you kill the parent, all children processes are killed off the next
> time they try to
> hearbeat. All of this happens transparently to you as a user.
>
> The new release comes with a brand new job scheduler. You can use
> simple triggers like repeat_every or you can get more complex and use
> full cron compatible syntax.
>
> Workers can be scheduled by two built in ''trigger'' types.
A simple
> ''trigger'' is specified with start, stop and interval:
>
>    require ''active_support''
>    MiddleMan.schedule_worker(
>      :class => :example_worker,
>      :job_key => :schedule_test,
>      :worker_method => :other_method,
>      :trigger_args => {
>        :start => Time.now + 5.seconds,
>        :end => Time.now + 10.minutes,
>        :repeat_interval => 30.seconds
>      }
>
> The cron trigger uses a similar syntax to cron found on UNIX systems:
>
>    MiddleMan.schedule_worker(
>      :class => simple_class,
>      :job_key => :schedule_test,
>      :worker_method => :arg_method,
>      :worker_method_args => "my argument to arg_method",
>      :trigger_args => "0 15 10 * * * *"
>    )
>
>
> Also note that when the server starts up, you will see 3 processes
> running.
> One of the is the MiddleMan server, one is the results worker and  
> one is
> the logger worker. When you do a logger.info("foo log!") in your
> workers,
> you are actually logging to the Logger worker.
>
> As you might imagine, this new way of managing multiple processes will
> scale a lot better then the multi threaded single process  
> version ;) But
> also be aware that there is still a thread pool in the middleman
> that  you
> can control the size of. All this does is keep the plugin from
> spawning too
> many processes. It will allow however many workers you specify to  
> run at
> once and any more then that will just queue up and wit for their  
> turn to
> spawn.
>
> If you are already a user of the old BackgrounDRb please give the new
> version a tryout. It should not require very much work to port your
> worker
> classes to the new architecture.
>
> This is alpha software folks. It works for me? but we will not be
> held responsible
> if it ruins your day ;)
>
> -- Ezra Zygmuntowicz 
> -- Lead Rails Evangelist
> -- ez at engineyard.com
> -- Engine Yard, Serious Rails Hosting
> -- (866) 518-YARD (9273)
>
>
>
>
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
-----------------------------
Siebert Michael
info at siebert-wd.de

Fantastic news! And perfect timing too, as I am beginning a very  
large project that will rely heavily on Backgroundrb and will gladly  
help root out bugs.

I did hit one snag in getting it up and running. I removed the  
current version of BackgroundRB and installed the trunk.

I ran setup and then backgroundrb start and received the following  
error:

$ script/backgroundrb start
ERROR: there is already one or more instance(s) of the program running

However there was no BackgroundRB running. I''m on OSX using daemons  
(1.0.1) and slave (1.0.0). I couldn''t find that the error message in  
the backgroundrb source.

Any thoughts?

Erik
On Oct 29, 2006, at 8:40 PM, Ezra Zygmuntowicz wrote:
> Hello Folks-
>
> 	I am really happy to announce the release of BackgrounDRb 0.2.0 .
> This is a complete rewrite and re-architecture of this codebase. I
> want to
> thank skaar for all of his hard work. I put out a call for help with
> the new
>   architecture and skaar stepped up bigtime. I had already written
> the new
> scheduler and cron compatible syntax and moved to multi process
> architecture. But he took what I had that wasn''t very whole and
did a
> whopping 100 svn commits to the project! Big ups to skaar for this
> release!
>
>
> = BackgrounDRb 0.2.0
>
>
> !! Note that this is in an entirely different svn repo then the old
> plugin.
> Make sure you remove the old before you install the new.
>
> http://svn.devjavu.com/backgroundrb/tags/release-0.2.0/README
> http://svn.devjavu.com/backgroundrb/tags/release-0.2.0 (latest  
> release)
> http://svn.devjavu.com/backgroundrb/trunk (svn trunk)
> http://backgroundrb.devjavu.com (trac)
> http://backgroundrb.rubyforge.org (rdoc)
>
> Copyright (c) 2006 Ezra Zygmuntowicz and skaar[at]waste[dot]org
>
> == DISCLAIMER
>
> The 0.2.0 release of BackgrounDRb is a complete re-write of previous
> releases and is to be considered experimental, in-complete and in many
> respect untested. Our goal is to reach a release recommended for
> production use by 0.3.x. This said, this release is a more robust
> solution that the previous release.
>
> Also note that support for Windows is deprecated with this release of
> BackgrounDRb, sorry : ( . It may work under cygwin but we have not
> tried it.
>
> == Introduction
>
> BackgrounDRb is a ruby job server and scheduler. It main intent is  
> to be
> used with Ruby on Rails applications for offloading long running  
> tasks.
> Since a rails application blocks while servicing a request it is best
> to move long running tasks off into a background process that is
> divorced
> from the http request/response cycle.
>
> This new release of BackgrounDRb is also modular and can be used  
> without
> Rails. So any ruby program or framework can use it.
>
> == Technology Overview
>
> This 0.2.x branch of BackgrounDRb introduces a completely new
> architecture
> then the previous versions. Instead of a single process, multi
> threaded environment,
> the new system uses multi process with IPC to manage workers. So each
> of your
> workers will run in their own ruby interpreter. The interface that you
> use within rails remains mostly unchanged.
>
> The new architecture allows for a much more robust system. Each of  
> your
> worker classes get run in their own ruby interpreter. The main server
> process
> holds a collection of references to your running worker objects. This
> acts as
> the interface or MiddleMan that you use to interact with your remote
> jobs from
> rails or other ruby client code. When workers are spawned an
> interprocess
> communication channel is opened between the MiddleMan and the worker.
> There is a heartbeat setup between parent and child processes so  
> that if
> you kill the parent, all children processes are killed off the next
> time they try to
> hearbeat. All of this happens transparently to you as a user.
>
> The new release comes with a brand new job scheduler. You can use
> simple triggers like repeat_every or you can get more complex and use
> full cron compatible syntax.
>
> Workers can be scheduled by two built in ''trigger'' types.
A simple
> ''trigger'' is specified with start, stop and interval:
>
>    require ''active_support''
>    MiddleMan.schedule_worker(
>      :class => :example_worker,
>      :job_key => :schedule_test,
>      :worker_method => :other_method,
>      :trigger_args => {
>        :start => Time.now + 5.seconds,
>        :end => Time.now + 10.minutes,
>        :repeat_interval => 30.seconds
>      }
>
> The cron trigger uses a similar syntax to cron found on UNIX systems:
>
>    MiddleMan.schedule_worker(
>      :class => simple_class,
>      :job_key => :schedule_test,
>      :worker_method => :arg_method,
>      :worker_method_args => "my argument to arg_method",
>      :trigger_args => "0 15 10 * * * *"
>    )
>
>
> Also note that when the server starts up, you will see 3 processes
> running.
> One of the is the MiddleMan server, one is the results worker and  
> one is
> the logger worker. When you do a logger.info("foo log!") in your
> workers,
> you are actually logging to the Logger worker.
>
> As you might imagine, this new way of managing multiple processes will
> scale a lot better then the multi threaded single process  
> version ;) But
> also be aware that there is still a thread pool in the middleman
> that  you
> can control the size of. All this does is keep the plugin from
> spawning too
> many processes. It will allow however many workers you specify to  
> run at
> once and any more then that will just queue up and wit for their  
> turn to
> spawn.
>
> If you are already a user of the old BackgrounDRb please give the new
> version a tryout. It should not require very much work to port your
> worker
> classes to the new architecture.
>
> This is alpha software folks. It works for me? but we will not be
> held responsible
> if it ruins your day ;)
>
> -- Ezra Zygmuntowicz 
> -- Lead Rails Evangelist
> -- ez at engineyard.com
> -- Engine Yard, Serious Rails Hosting
> -- (866) 518-YARD (9273)
>
>
>
>
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel

> PS: wheres the restart rake task gone? :)
it''s just one of several things that was in the old release that we
didn''t get around to re-add - please report these in trac at:

   http://backgroundrb.devjavu.com/ (you will have to register)

and we''ll get to them as soon as we can - there will be at least one
more 0.2.x release before we reach 0.3.x which would be the release we
expect to be "production ready" ( whatever that means :) )

All feedback and testing is much appreciated.

/skaar

-- 
----------------------------------------------------------------------
|\|\             where in the       |          s_u_b_s_t_r_u_c_t_i_o_n 
| | >===========  W.A.S.T.E.        |                  genarratologies 
|/|/    (_)     is the wisdom       |                  skaar at waste.org
----------------------------------------------------------------------

> However there was no BackgroundRB running. I''m on OSX using
daemons
> (1.0.1) and slave (1.0.0). I couldn''t find that the error message
in
> the backgroundrb source.
Ezra had issues with daemons pre 1.0.2 on MacOS X - which is pretty much
why we said 1.0.2 as a dependency in the readme - give it a shot.

/skaar

Erik-

	That error means that there is an old log/backgroundrb.pid file. If  
you remove it then that error should go away.

-Ezra

On Oct 30, 2006, at 8:19 AM, Erik Morton wrote:
> Fantastic news! And perfect timing too, as I am beginning a very
> large project that will rely heavily on Backgroundrb and will gladly
> help root out bugs.
>
> I did hit one snag in getting it up and running. I removed the
> current version of BackgroundRB and installed the trunk.
>
> I ran setup and then backgroundrb start and received the following
> error:
>
> $ script/backgroundrb start
> ERROR: there is already one or more instance(s) of the program running
>
> However there was no BackgroundRB running. I''m on OSX using
daemons
> (1.0.1) and slave (1.0.0). I couldn''t find that the error message
in
> the backgroundrb source.
>
> Any thoughts?
>
> Erik
> On Oct 29, 2006, at 8:40 PM, Ezra Zygmuntowicz wrote:
>
>> Hello Folks-
>>
>> 	I am really happy to announce the release of BackgrounDRb 0.2.0 .
>> This is a complete rewrite and re-architecture of this codebase. I
>> want to
>> thank skaar for all of his hard work. I put out a call for help with
>> the new
>>   architecture and skaar stepped up bigtime. I had already written
>> the new
>> scheduler and cron compatible syntax and moved to multi process
>> architecture. But he took what I had that wasn''t very whole
and did a
>> whopping 100 svn commits to the project! Big ups to skaar for this
>> release!
>>
>>
>> = BackgrounDRb 0.2.0
>>
>>
>> !! Note that this is in an entirely different svn repo then the old
>> plugin.
>> Make sure you remove the old before you install the new.
>>
>> http://svn.devjavu.com/backgroundrb/tags/release-0.2.0/README
>> http://svn.devjavu.com/backgroundrb/tags/release-0.2.0 (latest
>> release)
>> http://svn.devjavu.com/backgroundrb/trunk (svn trunk)
>> http://backgroundrb.devjavu.com (trac)
>> http://backgroundrb.rubyforge.org (rdoc)
>>
>> Copyright (c) 2006 Ezra Zygmuntowicz and skaar[at]waste[dot]org
>>
>> == DISCLAIMER
>>
>> The 0.2.0 release of BackgrounDRb is a complete re-write of previous
>> releases and is to be considered experimental, in-complete and in  
>> many
>> respect untested. Our goal is to reach a release recommended for
>> production use by 0.3.x. This said, this release is a more robust
>> solution that the previous release.
>>
>> Also note that support for Windows is deprecated with this release of
>> BackgrounDRb, sorry : ( . It may work under cygwin but we have not
>> tried it.
>>
>> == Introduction
>>
>> BackgrounDRb is a ruby job server and scheduler. It main intent is
>> to be
>> used with Ruby on Rails applications for offloading long running
>> tasks.
>> Since a rails application blocks while servicing a request it is best
>> to move long running tasks off into a background process that is
>> divorced
>> from the http request/response cycle.
>>
>> This new release of BackgrounDRb is also modular and can be used
>> without
>> Rails. So any ruby program or framework can use it.
>>
>> == Technology Overview
>>
>> This 0.2.x branch of BackgrounDRb introduces a completely new
>> architecture
>> then the previous versions. Instead of a single process, multi
>> threaded environment,
>> the new system uses multi process with IPC to manage workers. So each
>> of your
>> workers will run in their own ruby interpreter. The interface that  
>> you
>> use within rails remains mostly unchanged.
>>
>> The new architecture allows for a much more robust system. Each of
>> your
>> worker classes get run in their own ruby interpreter. The main server
>> process
>> holds a collection of references to your running worker objects. This
>> acts as
>> the interface or MiddleMan that you use to interact with your remote
>> jobs from
>> rails or other ruby client code. When workers are spawned an
>> interprocess
>> communication channel is opened between the MiddleMan and the worker.
>> There is a heartbeat setup between parent and child processes so
>> that if
>> you kill the parent, all children processes are killed off the next
>> time they try to
>> hearbeat. All of this happens transparently to you as a user.
>>
>> The new release comes with a brand new job scheduler. You can use
>> simple triggers like repeat_every or you can get more complex and use
>> full cron compatible syntax.
>>
>> Workers can be scheduled by two built in ''trigger''
types. A simple
>> ''trigger'' is specified with start, stop and interval:
>>
>>    require ''active_support''
>>    MiddleMan.schedule_worker(
>>      :class => :example_worker,
>>      :job_key => :schedule_test,
>>      :worker_method => :other_method,
>>      :trigger_args => {
>>        :start => Time.now + 5.seconds,
>>        :end => Time.now + 10.minutes,
>>        :repeat_interval => 30.seconds
>>      }
>>
>> The cron trigger uses a similar syntax to cron found on UNIX systems:
>>
>>    MiddleMan.schedule_worker(
>>      :class => simple_class,
>>      :job_key => :schedule_test,
>>      :worker_method => :arg_method,
>>      :worker_method_args => "my argument to arg_method",
>>      :trigger_args => "0 15 10 * * * *"
>>    )
>>
>>
>> Also note that when the server starts up, you will see 3 processes
>> running.
>> One of the is the MiddleMan server, one is the results worker and
>> one is
>> the logger worker. When you do a logger.info("foo log!") in
your
>> workers,
>> you are actually logging to the Logger worker.
>>
>> As you might imagine, this new way of managing multiple processes  
>> will
>> scale a lot better then the multi threaded single process
>> version ;) But
>> also be aware that there is still a thread pool in the middleman
>> that  you
>> can control the size of. All this does is keep the plugin from
>> spawning too
>> many processes. It will allow however many workers you specify to
>> run at
>> once and any more then that will just queue up and wit for their
>> turn to
>> spawn.
>>
>> If you are already a user of the old BackgrounDRb please give the new
>> version a tryout. It should not require very much work to port your
>> worker
>> classes to the new architecture.
>>
>> This is alpha software folks. It works for me? but we will not be
>> held responsible
>> if it ruins your day ;)
>>
>> -- Ezra Zygmuntowicz 
>> -- Lead Rails Evangelist
>> -- ez at engineyard.com
>> -- Engine Yard, Serious Rails Hosting
>> -- (866) 518-YARD (9273)
>>
>>
>>
>>
>> _______________________________________________
>> Backgroundrb-devel mailing list
>> Backgroundrb-devel at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>
-- Ezra Zygmuntowicz 
-- Lead Rails Evangelist
-- ez at engineyard.com
-- Engine Yard, Serious Rails Hosting
-- (866) 518-YARD (9273)

Hi,

On 29-Oct-06, at 8:40 PM, Ezra Zygmuntowicz wrote:
> 	I am really happy to announce the release of BackgrounDRb 0.2.0 .

This is good news.

Say you were running a few background processes and someone or  
something killed your main process. Is there any way of knowing which  
processes died because of heart failure when the server is restarted?

Cheers,
Bob

----
Bob Hutchison                  -- blogs at <http://www.recursive.ca/ 
hutch/>
Recursive Design Inc.          -- <http://www.recursive.ca/>
Raconteur                      -- <http://www.raconteur.info/>
xampl for Ruby                 -- <http://rubyforge.org/projects/xampl/>

> Say you were running a few background processes and someone or  
> something killed your main process. Is there any way of knowing which  
> processes died because of heart failure when the server is restarted?
The child processes should died when the main process dies. The workers
are created using the Slave library, which sets up a heartbeat in the
child. We don''t persist which workers was running. What is it that you
are trying to do? 

/skaar


-- 
----------------------------------------------------------------------
|\|\             where in the       |          s_u_b_s_t_r_u_c_t_i_o_n 
| | >===========  W.A.S.T.E.        |                  genarratologies 
|/|/    (_)     is the wisdom       |                  skaar at waste.org
----------------------------------------------------------------------

On Oct 30, 2006, at 11:46 AM, Bob Hutchison wrote:
> Hi,
>
> On 29-Oct-06, at 8:40 PM, Ezra Zygmuntowicz wrote:
>
>> 	I am really happy to announce the release of BackgrounDRb 0.2.0 .
>
>
> This is good news.
>
> Say you were running a few background processes and someone or  
> something killed your main process. Is there any way of knowing  
> which processes died because of heart failure when the server is  
> restarted?
>
> Cheers,
> Bob
>
\

Hey bob-

	Thats a good question. The current answer is no, if the middleman  
server gets killed, all workers will get killed as well. But looking  
to the future we will try to have a way to have dual middlemen so  
they can watch each other and bring the other one up if it dies. And  
maybe a way to serialize the current jobs to disk like journalling so  
if things crash they could be restored easily. Please do give me your  
thoughts on this. We are going to be working on this thing to make it  
awesome by 0.3.0 so if there are things you want to see then speak up.

Thanks
-- Ezra Zygmuntowicz 
-- Lead Rails Evangelist
-- ez at engineyard.com
-- Engine Yard, Serious Rails Hosting
-- (866) 518-YARD (9273)

On 30-Oct-06, at 3:01 PM, Ezra Zygmuntowicz wrote:
>
> On Oct 30, 2006, at 11:46 AM, Bob Hutchison wrote:
>
>> Hi,
>>
>> On 29-Oct-06, at 8:40 PM, Ezra Zygmuntowicz wrote:
>>
>>> 	I am really happy to announce the release of BackgrounDRb 0.2.0 .
>>
>>
>> This is good news.
>>
>> Say you were running a few background processes and someone or  
>> something killed your main process. Is there any way of knowing  
>> which processes died because of heart failure when the server is  
>> restarted?
>>
>> Cheers,
>> Bob
>>
> \
>
> Hey bob-
>
> 	Thats a good question. The current answer is no, if the middleman  
> server gets killed, all workers will get killed as well. But  
> looking to the future we will try to have a way to have dual  
> middlemen so they can watch each other and bring the other one up  
> if it dies. And maybe a way to serialize the current jobs to disk  
> like journalling so if things crash they could be restored easily.  
> Please do give me your thoughts on this. We are going to be working  
> on this thing to make it awesome by 0.3.0 so if there are things  
> you want to see then speak up.
I''m using slave 0.2.0 in my application right now. I''m using
it to
run some potentially very long running processes. In my case, it  
doesn''t really matter if they fail, but if they do I''d like
know
about it so I can tell the user and possibly re-start them. At the  
moment if that scenario occurs then I can only tell by a time out. If  
I wasn''t so lazy I could do something smarter, like mark the fact  
that the process is running somewhere in the DB (which might really  
be the filesystem), maybe with a pid. When I check on progress I can  
see if slave/backgroundrb has any knowledge of that process, if it  
doesn''t then I can assume that the server failed and as re-started.  
If slave/backgroundrb knows of the process but the process no longer  
exists then I can assume that it failed in an unpleasant way.

Anyway, that''s the kind of thing I had in mind.

Cheers,
Bob
>
> Thanks
> -- Ezra Zygmuntowicz-- Lead Rails Evangelist
> -- ez at engineyard.com
> -- Engine Yard, Serious Rails Hosting
> -- (866) 518-YARD (9273)
>
>
----
Bob Hutchison                  -- blogs at <http://www.recursive.ca/ 
hutch/>
Recursive Design Inc.          -- <http://www.recursive.ca/>
Raconteur                      -- <http://www.raconteur.info/>
xampl for Ruby                 -- <http://rubyforge.org/projects/xampl/>

> I''m using slave 0.2.0 in my application right now. I''m
using it to
> run some potentially very long running processes. In my case, it  
> doesn''t really matter if they fail, but if they do I''d
like know
> about it so I can tell the user and possibly re-start them. At the  
> moment if that scenario occurs then I can only tell by a time out. If  
> I wasn''t so lazy I could do something smarter, like mark the fact
> that the process is running somewhere in the DB (which might really  
> be the filesystem), maybe with a pid. When I check on progress I can  
> see if slave/backgroundrb has any knowledge of that process, if it  
> doesn''t then I can assume that the server failed and as
re-started.
> If slave/backgroundrb knows of the process but the process no longer  
> exists then I can assume that it failed in an unpleasant way.
the crude way of doing this right now is to use a singleton/named worker
- and call new_worker(:class => :foo_worker, :job_key => :job_name)
every time you access it. If the worker exist, you will just get the key
back, if it doesn''t the server will create the worker, and then return
the key.

You can the same way, with generated keys, on the client side initially
call new_worker without the :job_key argument and store the key - if you
next time have a key, you can call new_worker with the job_key that was
generated for you and that you, say, have stored in the users session.

/skaar


-- 
----------------------------------------------------------------------
|\|\             where in the       |          s_u_b_s_t_r_u_c_t_i_o_n 
| | >===========  W.A.S.T.E.        |                  genarratologies 
|/|/    (_)     is the wisdom       |                  skaar at waste.org
----------------------------------------------------------------------

On 30-Oct-06, at 6:32 PM, skaar wrote:
>> I''m using slave 0.2.0 in my application right now.
I''m using it to
>> run some potentially very long running processes. In my case, it
>> doesn''t really matter if they fail, but if they do
I''d like know
>> about it so I can tell the user and possibly re-start them. At the
>> moment if that scenario occurs then I can only tell by a time out. If
>> I wasn''t so lazy I could do something smarter, like mark the
fact
>> that the process is running somewhere in the DB (which might really
>> be the filesystem), maybe with a pid. When I check on progress I can
>> see if slave/backgroundrb has any knowledge of that process, if it
>> doesn''t then I can assume that the server failed and as
re-started.
>> If slave/backgroundrb knows of the process but the process no longer
>> exists then I can assume that it failed in an unpleasant way.
>
> the crude way of doing this right now is to use a singleton/named  
> worker
> - and call new_worker(:class => :foo_worker, :job_key => :job_name)
> every time you access it. If the worker exist, you will just get  
> the key
> back, if it doesn''t the server will create the worker, and then
return
> the key.
>
> You can the same way, with generated keys, on the client side  
> initially
> call new_worker without the :job_key argument and store the key -  
> if you
> next time have a key, you can call new_worker with the job_key that  
> was
> generated for you and that you, say, have stored in the users session.
Thanks, I can probably do something with that. So much for being  
lazy :-)

Cheers,
Bob
>
> /skaar
>
>
> -- 
> ----------------------------------------------------------------------
> |\|\             where in the       |          s_u_b_s_t_r_u_c_t_i_o_n
> | | >===========  W.A.S.T.E.        |                  genarratologies
> |/|/    (_)     is the wisdom       |                  skaar at waste.org
> ----------------------------------------------------------------------
----
Bob Hutchison                  -- blogs at <http://www.recursive.ca/ 
hutch/>
Recursive Design Inc.          -- <http://www.recursive.ca/>
Raconteur                      -- <http://www.raconteur.info/>
xampl for Ruby                 -- <http://rubyforge.org/projects/xampl/>