Guys, Is there any sort of mechanism for canceling workers? I''ve looked through the docs and don''t see any but may be overlooking. Thanks! John -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080217/7b15c6bd/attachment.html
On Sun, Feb 17, 2008 at 11:08 PM, John Wells <lists at sourceillustrated.com> wrote:> Guys, > > Is there any sort of mechanism for canceling workers? I''ve looked through > the docs and don''t see any but may be overlooking. >What you mean by canceling the job? You mean the one scheduled or one from queue?
On Feb 18, 2008 8:45 AM, hemant <gethemant at gmail.com> wrote:> On Sun, Feb 17, 2008 at 11:08 PM, John Wells > <lists at sourceillustrated.com> wrote: > > Guys, > > > > Is there any sort of mechanism for canceling workers? I''ve looked > through > > the docs and don''t see any but may be overlooking. > > > > What you mean by canceling the job? You mean the one scheduled or one > from queue?Well, when I sent this, it was to cancel a job that was started on an autostarted worker with ask_work. However, I''ve since changed the worker to not autostart. I then do call: # controller action to start def start_work MiddleMan.new_worker(:worker=>"my_worker") MiddleMan.ask_work(:worker=>"my_worker", :worker_method=>"my_method") end # controller action to stop def stop_work MiddleMan.delete_worker(:worker=>"my_worker") end This seems to work, but there are a few undesired things left dangling, as I haven''t found a way to "catch" the delete request and do a cleanup. Assuming this is impossible, is there any way to clear the last status registered by the worker with register_status if that worker is deleted? And, is the above the proper way of cancelling work? Thanks very much for your help! John <http://www.greatworx.com> -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080218/4d0b8380/attachment.html
On Feb 18, 2008 12:59 PM, John Wells <lists at sourceillustrated.com> wrote:> This seems to work, but there are a few undesired things left dangling, as > I haven''t found a way to "catch" the delete request and do a cleanup. > Assuming this is impossible, is there any way to clear the last status > registered by the worker with register_status if that worker is deleted? > > And, is the above the proper way of cancelling work? > > Thanks very much for your help! >Sorry to bump my own message, but I''m still very interested in learning if I''m approaching the above properly, and if there''s a way to clear a worker''s status when that worker is deleted. Thanks! John -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080220/378d5470/attachment.html
On Wed, Feb 20, 2008 at 11:50 PM, John Wells <lists at sourceillustrated.com> wrote:> > > > On Feb 18, 2008 12:59 PM, John Wells <lists at sourceillustrated.com> wrote: > > > > > > This seems to work, but there are a few undesired things left dangling, as > I haven''t found a way to "catch" the delete request and do a cleanup. > Assuming this is impossible, is there any way to clear the last status > registered by the worker with register_status if that worker is deleted? > > > > And, is the above the proper way of cancelling work? > > > > Thanks very much for your help! > > > > Sorry to bump my own message, but I''m still very interested in learning if > I''m approaching the above properly, and if there''s a way to clear a worker''s > status when that worker is deleted. >Hmm, No problems. The way you are deleting/canceling running workers is correct. But apparently, you want a callback, which should be invoked when worker dies, so you can cleanup stuff. Now, unfortunately this means that like all requests, request to kill a worker must be handled by the worker itself, but if a worker is going to handle a request to kill itself ( in another words calls "exit" from inside worker code ), then when you invoke: MiddleMan.delete_worker and worker is doing some processing then worker won''t handle delete_worker request until its done with processing whatever its doing. That was how "delete_worker" used to work earlier. I mean when you invoke delete_worker it used to invoke "exit" method on worker and worker used to exit as usual. But later we changed the code and we thought when you invoke delete_worker, your worker should exit immediately and request to kill a worker shouldn''t be queued. In current code, apart from executing "delete_worker" another way to kill workers is to call "exit" from inside of a worker. This will give you opportunity to do proper cleanup and stuff. I am not sure, if this is what you want, but "delete_worker" is like "kill -9 " of unix. So, if you want to handle graceful exit, call exit from worker code itself.
On Feb 20, 2008 2:22 PM, hemant <gethemant at gmail.com> wrote:> But later we changed the code and we thought when you invoke > delete_worker, your worker should exit immediately and request to kill > a worker shouldn''t be queued. > > In current code, apart from executing "delete_worker" another way to > kill workers is to call "exit" from inside of a worker. This will give > you opportunity to do proper cleanup and stuff. > I am not sure, if this is what you want, but "delete_worker" is like > "kill -9 " of unix. > > So, if you want to handle graceful exit, call exit from worker code > itself. >Thanks for your reply. So how would I be able to signal to the worker that it should call exit? I guess that is the challenge. The worker is involved in a large batch job, and I need to be able to send a message to it to tell it to kill itself. I had hacked out an approach using both a database table in which it would check for the existence of a certain row, and if there remove and kill itself, but I wondered if the framework itself provided a better way? Also, assuming I just continue to use delete_worker in the way I am today, is there any way possible to delete the last status the worker reported? Thanks for your help! John -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080220/9604e712/attachment.html
On Thu, Feb 21, 2008 at 1:02 AM, John Wells <lists at sourceillustrated.com> wrote:> > > On Feb 20, 2008 2:22 PM, hemant <gethemant at gmail.com> wrote: > > But later we changed the code and we thought when you invoke > > delete_worker, your worker should exit immediately and request to kill > > a worker shouldn''t be queued. > > > > In current code, apart from executing "delete_worker" another way to > > kill workers is to call "exit" from inside of a worker. This will give > > you opportunity to do proper cleanup and stuff. > > I am not sure, if this is what you want, but "delete_worker" is like > > "kill -9 " of unix. > > > > So, if you want to handle graceful exit, call exit from worker code > itself. > > > > Thanks for your reply. > > So how would I be able to signal to the worker that it should call exit? I > guess that is the challenge. The worker is involved in a large batch job, > and I need to be able to send a message to it to tell it to kill itself. I > had hacked out an approach using both a database table in which it would > check for the existence of a certain row, and if there remove and kill > itself, but I wondered if the framework itself provided a better way? > > Also, assuming I just continue to use delete_worker in the way I am today, > is there any way possible to delete the last status the worker reported? >Something like this: class FooWorker < BackgrounDRb::MetaWorker def some_work(args = nil) # do some job processing from database exit end end
On Feb 20, 2008 2:39 PM, hemant <gethemant at gmail.com> wrote:> On Thu, Feb 21, 2008 at 1:02 AM, John Wells <lists at sourceillustrated.com> > wrote: > > > > > > On Feb 20, 2008 2:22 PM, hemant <gethemant at gmail.com> wrote: > > > But later we changed the code and we thought when you invoke > > > delete_worker, your worker should exit immediately and request to kill > > > a worker shouldn''t be queued. > > > > > > In current code, apart from executing "delete_worker" another way to > > > kill workers is to call "exit" from inside of a worker. This will give > > > you opportunity to do proper cleanup and stuff. > > > I am not sure, if this is what you want, but "delete_worker" is like > > > "kill -9 " of unix. > > > > > > So, if you want to handle graceful exit, call exit from worker code > > itself. > > > > > > > Thanks for your reply. > > > > So how would I be able to signal to the worker that it should call exit? > I > > guess that is the challenge. The worker is involved in a large batch > job, > > and I need to be able to send a message to it to tell it to kill itself. > I > > had hacked out an approach using both a database table in which it would > > check for the existence of a certain row, and if there remove and kill > > itself, but I wondered if the framework itself provided a better way? > > > > Also, assuming I just continue to use delete_worker in the way I am > today, > > is there any way possible to delete the last status the worker reported? > > > > Something like this: > > class FooWorker < BackgrounDRb::MetaWorker > def some_work(args = nil) > # do some job processing from database > exit > end > endOk, but when would exit be called here? After all the work is done? I guess what I want is a kill -9 functionality, but with the ability to at the very least clear the worker''s status... -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080220/2526ffd7/attachment-0001.html
Hi all, Just and idea: The normal way of doing cleanup in Ruby is to use a "begin...ensure" construct. Maybe that will work here too? Try something like this: class FooWorker < BackgrounDRb::MetaWorker def some_work(args=nil) begin # Large batch job here ... rescue # (optional) # Error processing here ... ensure # Cleanup code here ... end end end Then, just invoke delete_worker when you need to, and if all is well, it should interrupt the running batch job, but your cleanup code *should* get executed before the worker finally dies. If it is not, I''d call it a BackgrounDRb bug, as it would be breaking some basic Ruby functionality. Let us know how it goes. Cheers, -- Yves-Eric Martin John Wells wrote:> > class FooWorker < BackgrounDRb::MetaWorker > def some_work(args = nil) > # do some job processing from database > exit > end > end > > > > Ok, but when would exit be called here? After all the work is done? > > I guess what I want is a kill -9 functionality, but with the ability > to at the very least clear the worker''s status... > ------------------------------------------------------------------------ > > _______________________________________________ > Backgroundrb-devel mailing list > Backgroundrb-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/backgroundrb-devel
On Wed, Feb 20, 2008 at 6:36 PM, Yves-Eric Martin < yem_backgroundrb-devel at filter.yve.net> wrote:> Then, just invoke delete_worker when you need to, and if all is well, it > should interrupt the running batch job, but your cleanup code *should* > get executed before the worker finally dies. If it is not, I''d call it a > BackgrounDRb bug, as it would be breaking some basic Ruby functionality. >Thanks! It''s a good idea, but unfortunately, I only want cleanup to happen when the worker is forceably deleted...not when it exits on its own. Ensure would cleanup in both cases. Thanks for your suggestion regardless. John -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080221/141dc389/attachment.html
John Wells wrote:> On Wed, Feb 20, 2008 at 6:36 PM, Yves-Eric Martin > <yem_backgroundrb-devel at filter.yve.net > <mailto:yem_backgroundrb-devel at filter.yve.net>> wrote: > > Thanks! It''s a good idea, but unfortunately, I only want cleanup to > happen when the worker is forceably deleted...not when it exits on its > own. Ensure would cleanup in both cases.Well that one is easily taken care of: just use a local variable to indicate whether the processing was interrupted or went to completion. You would probably simply use a boolean or constant integers to indicate the exit status, but I''ll write it here with a string for simplicity: class FooWorker < BackgrounDRb::MetaWorker def some_work(args=nil) job_exit_status = ''FORCEABLY_DELETED'' begin # Large batch job here ... rescue job_exit_status = ''ERRORED_OUT'' # Error processing here ... else job_exit_status = ''FINISHED_NORMALLY'' ensure case job_exit_status when ''FINISHED_NORMALLY'' # Your normal ending cleanup code here ... when ''ERRORED_OUT'' # Your error ending cleanup code here ... else # Your "delete_worker" ending cleanup code here ... end end end end Of course this still all assumes that delete_worker is *not* the "kill -9" hemant suggested, and that it will let your ensure clause run. Have you tried it out yet? Let us know how it goes. Cheers, -- Yves-Eric
On Wed, Feb 20, 2008 at 2:51 PM, John Wells <lists at sourceillustrated.com> wrote:> Ok, but when would exit be called here? After all the work is done? > > I guess what I want is a kill -9 functionality, but with the ability to at > the very least clear the worker''s status...Hemant, Would you mind answering this? I''m just getting going again after a nasty bout of the flu, but very interested in your answer. Thanks! John
Yves-Eric Martin
2008-Feb-24 01:21 UTC
[Backgroundrb-devel] Bug report: delete_worker breaks begin...ensure (was Re: Canceling work...)
Hi all, and welcome back John. John, you may have missed my second reply to you: http://rubyforge.org/pipermail/backgroundrb-devel/2008-February/001514.html Anyway, the bad news is: I just tried it out and it does not seem to work. delete_worker *is* the "kill -9" hemant suggested. Hemant, this is breaking some basic Ruby functionality, namely "begin...ensure". So can we please call this a bug? delete_worker should be a "kill", not a "kill -9", or at least not by default. We could add an argument to determine which behavior we want: # normal "kill" behavior: MiddleMan.delete_worker(:worker => :background_worker) # brutal "kill -9" behavior: MiddleMan.delete_worker(:worker => :background_worker, :kill => true) Here is first some code I used in the console to demonstrate in plain Ruby what should happen: ########################################################## def do_work(args=nil) job_exit_status = ''FORCED_EXIT'' puts "#{Time.now} -- do_work entered" begin # Large batch job here sleep 30 rescue job_exit_status = ''ERRORED_OUT'' # Error processing here else job_exit_status = ''FINISHED_NORMALLY'' ensure puts "#{Time.now} -- do_work exit status: #{job_exit_status}" end end ########################################################## If Ctrl-C is pressed before completion, here is what we get: >> do_work 2008-02-24 10:12:03 -- do_work entered 2008-02-24 10:12:04 -- do_work exit status: FORCED_EXIT Now here is the BackgrounDRb code used to show the bug: ########################################################## class BackgroundWorker < BackgrounDRb::MetaWorker set_worker_name :background_worker def create(args = nil); end def do_work(args=nil) job_exit_status = ''FORCEABLY_DELETED'' logger.info "#{Time.now} -- do_work entered" begin # Large batch job here sleep 30 rescue job_exit_status = ''ERRORED_OUT'' # Error processing here else job_exit_status = ''FINISHED_NORMALLY'' ensure logger.info "#{Time.now} -- do_work exit status: #{job_exit_status}" end end end ########################################################## If allowed to finish, we get this in the logs: 2008-02-24 09:45:37 -- do_work entered 2008-02-24 09:45:47 -- do_work exit status: FINISHED_NORMALLY However, if delete_worker is called before completion, we only get: 2008-02-24 09:46:03 -- do_work entered Cheers, -- Yves-Eric
Hi On Sun, Feb 24, 2008 at 3:01 AM, John Wells <lists at sourceillustrated.com> wrote:> On Wed, Feb 20, 2008 at 2:51 PM, John Wells <lists at sourceillustrated.com> wrote: > > Ok, but when would exit be called here? After all the work is done? > > > > I guess what I want is a kill -9 functionality, but with the ability to at > > the very least clear the worker''s status... > > Hemant, > > Would you mind answering this? I''m just getting going again after a > nasty bout of the flu, but very interested in your answer. >John, Sorry for later reply. Look into my reply in another thread by Eric.
hemant
2008-Feb-24 18:37 UTC
[Backgroundrb-devel] Bug report: delete_worker breaks begin...ensure (was Re: Canceling work...)
Hi, On Sun, Feb 24, 2008 at 6:51 AM, Yves-Eric Martin <yem_backgroundrb-devel at filter.yve.net> wrote:> Hi all, and welcome back John. > > > John, you may have missed my second reply to you: > http://rubyforge.org/pipermail/backgroundrb-devel/2008-February/001514.html > > Anyway, the bad news is: I just tried it out and it does not seem to > work. delete_worker *is* the "kill -9" hemant suggested. > > Hemant, this is breaking some basic Ruby functionality, namely > "begin...ensure". So can we please call this a bug? delete_worker should > be a "kill", not a "kill -9", or at least not by default. We could add > an argument to determine which behavior we want: > > # normal "kill" behavior: > MiddleMan.delete_worker(:worker => :background_worker) > > # brutal "kill -9" behavior: > MiddleMan.delete_worker(:worker => :background_worker, :kill => true) > > > > Here is first some code I used in the console to demonstrate in plain > Ruby what should happen: > > ########################################################## > def do_work(args=nil) > job_exit_status = ''FORCED_EXIT'' > puts "#{Time.now} -- do_work entered" > begin > # Large batch job here > sleep 30 > rescue > job_exit_status = ''ERRORED_OUT'' > # Error processing here > else > job_exit_status = ''FINISHED_NORMALLY'' > ensure > puts "#{Time.now} -- do_work exit status: #{job_exit_status}" > end > end > ########################################################## > > > If Ctrl-C is pressed before completion, here is what we get: > > >> do_work > 2008-02-24 10:12:03 -- do_work entered > 2008-02-24 10:12:04 -- do_work exit status: FORCED_EXIT > > > > Now here is the BackgrounDRb code used to show the bug: > > ########################################################## > class BackgroundWorker < BackgrounDRb::MetaWorker > set_worker_name :background_worker > def create(args = nil); end > def do_work(args=nil) > job_exit_status = ''FORCEABLY_DELETED'' > logger.info "#{Time.now} -- do_work entered" > begin > # Large batch job here > sleep 30 > rescue > job_exit_status = ''ERRORED_OUT'' > # Error processing here > else > job_exit_status = ''FINISHED_NORMALLY'' > ensure > logger.info "#{Time.now} -- do_work exit status: #{job_exit_status}" > end > end > end > ########################################################## > > If allowed to finish, we get this in the logs: > 2008-02-24 09:45:37 -- do_work entered > 2008-02-24 09:45:47 -- do_work exit status: FINISHED_NORMALLY > > However, if delete_worker is called before completion, we only get: > 2008-02-24 09:46:03 -- do_work enteredEric, Problem is if you look into code of delete_worker then, you will find that we are sending TERM and KILL signals to the worker and hence its not possible to use "ensure" there. Why, we have both "TERM" and "KILL" there is because, signals to kill a task is not cross platform across unixes. But if we remove "KILL" and have "TERM" only, your worker will work as you expected. Along with Alex patches, I am trying to push this change in git, so you can try. But, currently gitorious is down, sadly. I am literally waiting for getting the damn thing up.
On Sun, Feb 24, 2008 at 3:01 AM, John Wells <lists at sourceillustrated.com> wrote:> On Wed, Feb 20, 2008 at 2:51 PM, John Wells <lists at sourceillustrated.com> wrote: > > Ok, but when would exit be called here? After all the work is done? > > > > I guess what I want is a kill -9 functionality, but with the ability to at > > the very least clear the worker''s status... >And you shall have it. I am just not sure if it will work across all *nixes. It works on Linux and thats the only OS I have. As suggested by Eric, you can use "ensure" to do condition cleanup, if you don''t have "ensure" your worker die as usual.
hemant
2008-Feb-25 20:49 UTC
[Backgroundrb-devel] Bug report: delete_worker breaks begin...ensure (was Re: Canceling work...)
On Tue, Feb 26, 2008 at 1:06 AM, John Wells <john.wells at greatworx.com> wrote:> > On Sun, Feb 24, 2008 at 1:37 PM, hemant <gethemant at gmail.com> wrote: > > Hi, > > > > > > > > On Sun, Feb 24, 2008 at 6:51 AM, Yves-Eric Martin > > <yem_backgroundrb-devel at filter.yve.net> wrote: > > > Hi all, and welcome back John. > > > > > > > > > John, you may have missed my second reply to you: > > > http://rubyforge.org/pipermail/backgroundrb-devel/2008-February/001514.html > > > > > > Anyway, the bad news is: I just tried it out and it does not seem to > > > work. delete_worker *is* the "kill -9" hemant suggested. > > > > > > Hemant, this is breaking some basic Ruby functionality, namely > > > "begin...ensure". So can we please call this a bug? delete_worker should > > > be a "kill", not a "kill -9", or at least not by default. We could add > > > an argument to determine which behavior we want: > > > > > > # normal "kill" behavior: > > > MiddleMan.delete_worker(:worker => :background_worker) > > > > > > # brutal "kill -9" behavior: > > > MiddleMan.delete_worker(:worker => :background_worker, :kill => true) > > > > > > > > > > > > Here is first some code I used in the console to demonstrate in plain > > > Ruby what should happen: > > > > > > ########################################################## > > > def do_work(args=nil) > > > job_exit_status = ''FORCED_EXIT'' > > > puts "#{Time.now} -- do_work entered" > > > begin > > > # Large batch job here > > > sleep 30 > > > rescue > > > job_exit_status = ''ERRORED_OUT'' > > > # Error processing here > > > else > > > job_exit_status = ''FINISHED_NORMALLY'' > > > ensure > > > puts "#{Time.now} -- do_work exit status: #{job_exit_status}" > > > end > > > end > > > ########################################################## > > > > > > > > > If Ctrl-C is pressed before completion, here is what we get: > > > > > > >> do_work > > > 2008-02-24 10:12:03 -- do_work entered > > > 2008-02-24 10:12:04 -- do_work exit status: FORCED_EXIT > > > > > > > > > > > > Now here is the BackgrounDRb code used to show the bug: > > > > > > ########################################################## > > > class BackgroundWorker < BackgrounDRb::MetaWorker > > > set_worker_name :background_worker > > > def create(args = nil); end > > > def do_work(args=nil) > > > job_exit_status = ''FORCEABLY_DELETED'' > > > logger.info "#{Time.now} -- do_work entered" > > > begin > > > # Large batch job here > > > sleep 30 > > > rescue > > > job_exit_status = ''ERRORED_OUT'' > > > # Error processing here > > > else > > > job_exit_status = ''FINISHED_NORMALLY'' > > > ensure > > > logger.info "#{Time.now} -- do_work exit status: #{job_exit_status}" > > > end > > > end > > > end > > > ########################################################## > > > > > > If allowed to finish, we get this in the logs: > > > 2008-02-24 09:45:37 -- do_work entered > > > 2008-02-24 09:45:47 -- do_work exit status: FINISHED_NORMALLY > > > > > > However, if delete_worker is called before completion, we only get: > > > 2008-02-24 09:46:03 -- do_work entered > > > > Eric, > > > > Problem is if you look into code of delete_worker then, you will find > > that we are sending TERM and KILL signals to the worker and hence its > > not possible to use "ensure" there. > > > > Why, we have both "TERM" and "KILL" there is because, signals to kill > > a task is not cross platform across unixes. > > But if we remove "KILL" and have "TERM" only, your worker will work as > > you expected. > > > > Along with Alex patches, I am trying to push this change in git, so > > you can try. But, currently gitorious is down, sadly. I am literally > > waiting for getting the damn thing up. > > Thanks guys. I''m looking forward to seeing these patches.They are there in the git repo, you can get latest code using: git clone git://gitorious.org/backgroundrb/mainline.git backgroundrb Also, you need to do: sudo gem install packet chronic