I tried to upgrade my existing application to RC2 last night. Like many, I use this mostly for running scheduled tasks. For the moment, I''ve abandoned the effort, but am looking forward to being able to use this. Feedback below: First, the reason I was looking forward to this upgrade was to use the threaded scheduler. I have an application with long-running tasks. I found that scheduled tasks in previous versions of backgroundrb would not run if the worker scheduled before it was still running. This seemed counterintuitive to me (given the whole idea behind backgroundrb) but after reading documentation and forums, it seems many others ran into this issue too. RC2 planned to do away with this. The Good: - (Still) Easy Installattion - Well designed. Logging is centralized, that''s nice. - Easy to hack up for use in an enterprise environment. The Bad: - Lots of arbitrary changes that I don''t quite understand. Different function names to create workers, new YML formats, different config files for scheduler. Would have been great if none of the rails application-facing stuff changed so the new version could just be swapped in. - Logging detail has greatly diminished. It''s difficult to troubleshoot what''s happening when things go wrong. - For the moment, this seems less stable than the old version. My scheduled tasks are bringing down the whole server and I can''t tell why (nothing showing up in the logs). This is why I''ve abandoned the upgrade. The scheduler is working fine for lightweight tasks, but the long-running ones just bring everything down. For what it''s worth, the logic for the tasks still lives in my rails models. The workers basically just call class methods in the models. (Truly using this as a scheduler.) Perhaps this has something to do with it? - Documentation and resources for the new version are sparse (obviously) and seem a little inaccurate. For example, I learned that the MiddleMan.ask_worker() method detailed in the docs is actually called as MiddleMan.ask_work (). If anyone has any thoughts on what might be going wrong with my scheduled tasks, that''d be great. Thanks! Andy -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20071205/46073cc7/attachment.html
On Wed, 2007-12-05 at 17:51 -0800, Andy Tyra wrote:> I tried to upgrade my existing application to RC2 last night. Like > many, I use this mostly for running scheduled tasks. For the moment, > I''ve abandoned the effort, but am looking forward to being able to use > this. Feedback below: > > First, the reason I was looking forward to this upgrade was to use the > threaded scheduler. I have an application with long-running tasks. I > found that scheduled tasks in previous versions of backgroundrb would > not run if the worker scheduled before it was still running. This > seemed counterintuitive to me (given the whole idea behind > backgroundrb) but after reading documentation and forums, it seems > many others ran into this issue too. RC2 planned to do away with > this. > > The Good: > > - (Still) Easy Installattion > - Well designed. Logging is centralized, that''s nice. > - Easy to hack up for use in an enterprise environment. > > The Bad: > > - Lots of arbitrary changes that I don''t quite understand. Different > function names to create workers, new YML formats, different config > files for scheduler. Would have been great if none of the rails > application-facing stuff changed so the new version could just be > swapped in. > - Logging detail has greatly diminished. It''s difficult to > troubleshoot what''s happening when things go wrong. > - For the moment, this seems less stable than the old version. My > scheduled tasks are bringing down the whole server and I can''t tell > why (nothing showing up in the logs). This is why I''ve abandoned the > upgrade. The scheduler is working fine for lightweight tasks, but the > long-running ones just bring everything down. For what it''s worth, > the logic for the tasks still lives in my rails models. The workers > basically just call class methods in the models. (Truly using this as > a scheduler.) Perhaps this has something to do with it? > - Documentation and resources for the new version are sparse > (obviously) and seem a little inaccurate. For example, I learned that > the MiddleMan.ask_worker() method detailed in the docs is actually > called as MiddleMan.ask_work ().I also saw some typos in the documentation and hence fixing them.> > If anyone has any thoughts on what might be going wrong with my > scheduled tasks, that''d be great. Thanks!There is definitely an exception being thrown in your worker thats not handled. if possible, please wrap your worker method in a begin rescue and see what happens. Currently, before daemonizing workers, i am redirecting their STDOUT,STDIN and STDERR to a file, so as unhandled exceptions gets logged. I will get this better. Long running schedulers are most important thing and hence any bug will be fixed with atmost care. Also, regarding multi tasksing that you want to achieve. New release of bdrb basically encourages cooperative multitasking not preemtpive multitasking of threads.I will code some examples for this. Thanks for your patience and please try the fix i suggested.
Hi There, Thanks for the help! I have made some progress. It looks like the worker is crashing because it is not getting passed the argument that it requires to run. That would explain why kicking this worker off manually worked just fine, but doing so in the scheduler did not. The problem is, I need to be able to pass a parameter to this worker''s function using the scheduling YML and that doesn''t seem to be working. Perhaps you can help troubleshoot? Here is the relevant method from the worker which is named ExpireAndRefreshWorker: def execute_report(report_name) myReport = Report.new(report_name) begin File.delete (File.join(RAILS_ROOT, ''public'',''reports'', myReport.const_name + ''.html'')) rescue #do nothing end renderController = FakeTest.new renderController.render_report (myReport) end As you can see, it takes one parameter, report_name, which should be a string. Here''s the YML I''m trying to use to schedule it: :schedules: :expire_and_refresh_worker: :worker_method: execute_report :data: unfilled_demand :trigger_args: 0 */15 * * * * * :job_key: unfilled_demand_schedule And here''s the stack trace of the exception that is thrown when the schedule triggers: (Please note that the line numbers in my stack trace will not match yours for script/backgroundrb. I''ve added some additional tasks to this script that add YML scheduling rails-root/vendor/plugins/backgroundrb/server/meta_worker.rb:91:in `execute_report'': wrong number of arguments (0 for 1) (ArgumentError) from rails-root/vendor/plugins/backgroundrb/server/meta_worker.rb:91:in `send'' from rails-root/vendor/plugins/backgroundrb/server/meta_worker.rb:91:in `check_for_timer_events'' from rails-root/vendor/plugins/backgroundrb/framework/core.rb:129:in `start_reactor'' from rails-root/vendor/plugins/backgroundrb/framework/core.rb:128:in `loop'' from rails-root/vendor/plugins/backgroundrb/framework/core.rb:128:in `start_reactor'' from rails-root/vendor/plugins/backgroundrb/framework/worker.rb:21:in `start_worker'' from rails-root/vendor/plugins/backgroundrb/framework/packet_master.rb:152:in `fork_and_load'' from rails-root/vendor/plugins/backgroundrb/framework/packet_master.rb:111:in `load_workers'' from rails-root/vendor/plugins/backgroundrb/framework/packet_master.rb:106:in `each'' from rails-root/vendor/plugins/backgroundrb/framework/packet_master.rb:106:in `load_workers'' from rails-root/vendor/plugins/backgroundrb/framework/packet_master.rb:19:in `run'' from rails-root/vendor/plugins/backgroundrb/server/master_worker.rb:114:in `initialize'' from script/backgroundrb:73:in `new'' from script/backgroundrb:73 It appears the parameter specified as :data in the yml is not getting passed to the function. Is there some other way this should be done? I don''t see it in the doco. Is it done as :args like in the old version? Also, this is unrelated, but I just thought I''d point it out. The following stack trace appears in the logs when stopping backgroundrb: rails-root/config/../vendor/plugins/backgroundrb/lib/../framework/nbio.rb:20:in `read_data'': Packet::DisconnectError (Packet::DisconnectError) from rails-root/vendor/plugins/backgroundrb/framework/worker.rb:47:in `handle_internal_messages'' from rails-root/vendor/plugins/backgroundrb/framework/core.rb:138:in `start_reactor'' from rails-root/vendor/plugins/backgroundrb/framework/core.rb:136:in `each'' from rails-root/vendor/plugins/backgroundrb/framework/core.rb:136:in `start_reactor'' from rails-root/vendor/plugins/backgroundrb/framework/core.rb:128:in `loop'' from rails-root/vendor/plugins/backgroundrb/framework/core.rb:128:in `start_reactor'' from rails-root/vendor/plugins/backgroundrb/framework/worker.rb:21:in `start_worker'' from rails-root/vendor/plugins/backgroundrb/framework/packet_master.rb:152:in `fork_and_load'' from rails-root/vendor/plugins/backgroundrb/framework/packet_master.rb:128:in `start_worker'' from rails-root/vendor/plugins/backgroundrb/server/master_worker.rb:115:in `initialize'' from rails-root/vendor/plugins/backgroundrb/framework/packet_master.rb:18:in `run'' from rails-root/vendor/plugins/backgroundrb/server/master_worker.rb:114:in `initialize'' from script/backgroundrb:73:in `new'' from script/backgroundrb:73 And, a question. I think I''ve figured out where everything is getting logged. Will this version eventually log things like the old version? The old version would report when workers stopped and started, when schedules were loaded and triggered, etc. Is there somewhere you''d like this sort of thing submitted as a feature request? Thanks, Andy On Dec 5, 2007 6:54 PM, hemant kumar <gethemant at gmail.com> wrote:> > On Wed, 2007-12-05 at 17:51 -0800, Andy Tyra wrote: > > I tried to upgrade my existing application to RC2 last night. Like > > many, I use this mostly for running scheduled tasks. For the moment, > > I''ve abandoned the effort, but am looking forward to being able to use > > this. Feedback below: > > > > First, the reason I was looking forward to this upgrade was to use the > > threaded scheduler. I have an application with long-running tasks. I > > found that scheduled tasks in previous versions of backgroundrb would > > not run if the worker scheduled before it was still running. This > > seemed counterintuitive to me (given the whole idea behind > > backgroundrb) but after reading documentation and forums, it seems > > many others ran into this issue too. RC2 planned to do away with > > this. > > > > The Good: > > > > - (Still) Easy Installattion > > - Well designed. Logging is centralized, that''s nice. > > - Easy to hack up for use in an enterprise environment. > > > > The Bad: > > > > - Lots of arbitrary changes that I don''t quite understand. Different > > function names to create workers, new YML formats, different config > > files for scheduler. Would have been great if none of the rails > > application-facing stuff changed so the new version could just be > > swapped in. > > - Logging detail has greatly diminished. It''s difficult to > > troubleshoot what''s happening when things go wrong. > > - For the moment, this seems less stable than the old version. My > > scheduled tasks are bringing down the whole server and I can''t tell > > why (nothing showing up in the logs). This is why I''ve abandoned the > > upgrade. The scheduler is working fine for lightweight tasks, but the > > long-running ones just bring everything down. For what it''s worth, > > the logic for the tasks still lives in my rails models. The workers > > basically just call class methods in the models. (Truly using this as > > a scheduler.) Perhaps this has something to do with it? > > - Documentation and resources for the new version are sparse > > (obviously) and seem a little inaccurate. For example, I learned that > > the MiddleMan.ask_worker() method detailed in the docs is actually > > called as MiddleMan.ask_work (). > > I also saw some typos in the documentation and hence fixing them. > > > > If anyone has any thoughts on what might be going wrong with my > > scheduled tasks, that''d be great. Thanks! > > There is definitely an exception being thrown in your worker thats not > handled. if possible, please wrap your worker method in a begin rescue > and see what happens. Currently, before daemonizing workers, i am > redirecting their STDOUT,STDIN and STDERR to a file, so as unhandled > exceptions gets logged. I will get this better. Long running schedulers > are most important thing and hence any bug will be fixed with atmost > care. > > Also, regarding multi tasksing that you want to achieve. New release of > bdrb basically encourages cooperative multitasking not preemtpive > multitasking of threads.I will code some examples for this. > > Thanks for your patience and please try the fix i suggested. > > > > > >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20071206/8b7f86e9/attachment.html
Another issue to report. When running the aforementioned worker manually, it runs and seems to work, but the worker process crashes at the end, leaving this stack trace: rails-root/config/../vendor/plugins/backgroundrb/lib/../framework/nbio.rb:55:in `dump'': can''t dump anonymous class #<Class:0xb6a4ccc8> (TypeError) from rails-root/config/../vendor/plugins/backgroundrb/lib/../framework/nbio.rb:55:in `dump_object'' from rails-root/vendor/plugins/backgroundrb/framework/worker.rb:32:in `send_data'' from rails-root/vendor/plugins/backgroundrb/server/meta_worker.rb:79:in `send_response'' from rails-root/lib/workers/expire_and_refresh_worker.rb:36:in `process_request'' from rails-root/vendor/plugins/backgroundrb/server/meta_worker.rb:48:in `receive_data'' from rails-root/vendor/plugins/backgroundrb/framework/worker.rb:54:in `receive_internal_data'' from rails-root/config/../vendor/plugins/backgroundrb/lib/../framework/bin_parser.rb:30:in `extract'' from rails-root/vendor/plugins/backgroundrb/framework/worker.rb:52:in `receive_internal_data'' ... 11 levels... from rails-root/vendor/plugins/backgroundrb/framework/packet_master.rb:19:in `run'' from rails-root/vendor/plugins/backgroundrb/server/master_worker.rb:114:in `initialize'' from script/backgroundrb:73:in `new'' from script/backgroundrb:73 It looked like this was because my worker method was not returning anything, so I added the line return "I am done." to the end of the worker method. I am now getting this stack trace: rails-root/vendor/plugins/backgroundrb/framework/meta_pimp.rb:55:in `process_response'': You have a nil object when you didn''t expect it! (NoMethodError) The error occurred while evaluating nil.instance from rails-root/vendor/plugins/backgroundrb/framework/meta_pimp.rb:28:in `handle_object'' from rails-root/vendor/plugins/backgroundrb/framework/meta_pimp.rb:19:in `receive_data'' from rails-root/config/../vendor/plugins/backgroundrb/lib/../framework/bin_parser.rb:30:in `extract'' from rails-root/vendor/plugins/backgroundrb/framework/meta_pimp.rb:17:in `receive_data'' from rails-root/vendor/plugins/backgroundrb/framework/packet_master.rb:81:in `handle_internal_messages'' from rails-root/vendor/plugins/backgroundrb/framework/core.rb:138:in `start_reactor'' from rails-root/vendor/plugins/backgroundrb/framework/core.rb:136:in `each'' from rails-root/vendor/plugins/backgroundrb/framework/core.rb:136:in `start_reactor'' from rails-root/vendor/plugins/backgroundrb/framework/core.rb:128:in `loop'' from rails-root/vendor/plugins/backgroundrb/framework/core.rb:128:in `start_reactor'' from rails-root/vendor/plugins/backgroundrb/framework/packet_master.rb:20:in `run'' from rails-root/vendor/plugins/backgroundrb/server/master_worker.rb:114:in `initialize'' from script/backgroundrb:73:in `new'' from script/backgroundrb:73 On Dec 6, 2007 4:31 PM, Andy Tyra <andy.tyra at gmail.com> wrote:> Hi There, > > Thanks for the help! I have made some progress. > > It looks like the worker is crashing because it is not getting passed the > argument that it requires to run. That would explain why kicking this > worker off manually worked just fine, but doing so in the scheduler did > not. The problem is, I need to be able to pass a parameter to this worker''s > function using the scheduling YML and that doesn''t seem to be working. > Perhaps you can help troubleshoot? > > Here is the relevant method from the worker which is named > ExpireAndRefreshWorker: > > def execute_report(report_name) > myReport = Report.new(report_name) > begin > File.delete (File.join(RAILS_ROOT, ''public'',''reports'', > myReport.const_name + ''.html'')) > rescue > #do nothing > end > renderController = FakeTest.new > renderController.render_report (myReport) > end > > As you can see, it takes one parameter, report_name, which should be a > string. > > Here''s the YML I''m trying to use to schedule it: > > :schedules: > :expire_and_refresh_worker: > :worker_method: execute_report > :data: unfilled_demand > :trigger_args: 0 */15 * * * * * > :job_key: unfilled_demand_schedule > > And here''s the stack trace of the exception that is thrown when the > schedule triggers: (Please note that the line numbers in my stack trace > will not match yours for script/backgroundrb. I''ve added some additional > tasks to this script that add YML scheduling > > rails-root/vendor/plugins/backgroundrb/server/meta_worker.rb:91:in > `execute_report'': wrong number of arguments (0 for 1) (ArgumentError) > from > rails-root/vendor/plugins/backgroundrb/server/meta_worker.rb:91:in `send'' > from > rails-root/vendor/plugins/backgroundrb/server/meta_worker.rb:91:in > `check_for_timer_events'' > from > rails-root/vendor/plugins/backgroundrb/framework/core.rb:129:in > `start_reactor'' > from > rails-root/vendor/plugins/backgroundrb/framework/core.rb:128:in `loop'' > from > rails-root/vendor/plugins/backgroundrb/framework/core.rb:128:in > `start_reactor'' > from > rails-root/vendor/plugins/backgroundrb/framework/worker.rb:21:in > `start_worker'' > from > rails-root/vendor/plugins/backgroundrb/framework/packet_master.rb:152:in > `fork_and_load'' > from > rails-root/vendor/plugins/backgroundrb/framework/packet_master.rb:111:in > `load_workers'' > from > rails-root/vendor/plugins/backgroundrb/framework/packet_master.rb:106:in > `each'' > from > rails-root/vendor/plugins/backgroundrb/framework/packet_master.rb:106:in > `load_workers'' > from > rails-root/vendor/plugins/backgroundrb/framework/packet_master.rb:19:in > `run'' > from > rails-root/vendor/plugins/backgroundrb/server/master_worker.rb:114:in > `initialize'' > from script/backgroundrb:73:in `new'' > from script/backgroundrb:73 > > It appears the parameter specified as :data in the yml is not getting > passed to the function. Is there some other way this should be done? I > don''t see it in the doco. Is it done as :args like in the old version? > > Also, this is unrelated, but I just thought I''d point it out. The > following stack trace appears in the logs when stopping backgroundrb: > > rails-root/config/../vendor/plugins/backgroundrb/lib/../framework/nbio.rb:20:in > `read_data'': Packet::DisconnectError (Packet::DisconnectError) > from > rails-root/vendor/plugins/backgroundrb/framework/worker.rb:47:in > `handle_internal_messages'' > from > rails-root/vendor/plugins/backgroundrb/framework/core.rb:138:in > `start_reactor'' > from > rails-root/vendor/plugins/backgroundrb/framework/core.rb:136:in `each'' > from > rails-root/vendor/plugins/backgroundrb/framework/core.rb:136:in > `start_reactor'' > from > rails-root/vendor/plugins/backgroundrb/framework/core.rb:128:in `loop'' > from > rails-root/vendor/plugins/backgroundrb/framework/core.rb:128:in > `start_reactor'' > from > rails-root/vendor/plugins/backgroundrb/framework/worker.rb:21:in > `start_worker'' > from > rails-root/vendor/plugins/backgroundrb/framework/packet_master.rb:152:in > `fork_and_load'' > from > rails-root/vendor/plugins/backgroundrb/framework/packet_master.rb:128:in > `start_worker'' > from > rails-root/vendor/plugins/backgroundrb/server/master_worker.rb:115:in > `initialize'' > from > rails-root/vendor/plugins/backgroundrb/framework/packet_master.rb:18:in > `run'' > from > rails-root/vendor/plugins/backgroundrb/server/master_worker.rb:114:in > `initialize'' > from script/backgroundrb:73:in `new'' > from script/backgroundrb:73 > > And, a question. I think I''ve figured out where everything is getting > logged. Will this version eventually log things like the old version? The > old version would report when workers stopped and started, when schedules > were loaded and triggered, etc. Is there somewhere you''d like this sort of > thing submitted as a feature request? > > Thanks, > Andy > > > On Dec 5, 2007 6:54 PM, hemant kumar <gethemant at gmail.com> wrote: > > > > > On Wed, 2007-12-05 at 17:51 -0800, Andy Tyra wrote: > > > I tried to upgrade my existing application to RC2 last night. Like > > > many, I use this mostly for running scheduled tasks. For the moment, > > > I''ve abandoned the effort, but am looking forward to being able to use > > > this. Feedback below: > > > > > > First, the reason I was looking forward to this upgrade was to use the > > > threaded scheduler. I have an application with long-running tasks. I > > > > > found that scheduled tasks in previous versions of backgroundrb would > > > not run if the worker scheduled before it was still running. This > > > seemed counterintuitive to me (given the whole idea behind > > > backgroundrb) but after reading documentation and forums, it seems > > > many others ran into this issue too. RC2 planned to do away with > > > this. > > > > > > The Good: > > > > > > - (Still) Easy Installattion > > > - Well designed. Logging is centralized, that''s nice. > > > - Easy to hack up for use in an enterprise environment. > > > > > > The Bad: > > > > > > - Lots of arbitrary changes that I don''t quite understand. Different > > > function names to create workers, new YML formats, different config > > > files for scheduler. Would have been great if none of the rails > > > application-facing stuff changed so the new version could just be > > > swapped in. > > > - Logging detail has greatly diminished. It''s difficult to > > > troubleshoot what''s happening when things go wrong. > > > - For the moment, this seems less stable than the old version. My > > > scheduled tasks are bringing down the whole server and I can''t tell > > > why (nothing showing up in the logs). This is why I''ve abandoned the > > > upgrade. The scheduler is working fine for lightweight tasks, but the > > > > > long-running ones just bring everything down. For what it''s worth, > > > the logic for the tasks still lives in my rails models. The workers > > > basically just call class methods in the models. (Truly using this as > > > > > a scheduler.) Perhaps this has something to do with it? > > > - Documentation and resources for the new version are sparse > > > (obviously) and seem a little inaccurate. For example, I learned that > > > the MiddleMan.ask_worker() method detailed in the docs is actually > > > called as MiddleMan.ask_work (). > > > > I also saw some typos in the documentation and hence fixing them. > > > > > > If anyone has any thoughts on what might be going wrong with my > > > scheduled tasks, that''d be great. Thanks! > > > > There is definitely an exception being thrown in your worker thats not > > handled. if possible, please wrap your worker method in a begin rescue > > and see what happens. Currently, before daemonizing workers, i am > > redirecting their STDOUT,STDIN and STDERR to a file, so as unhandled > > exceptions gets logged. I will get this better. Long running schedulers > > are most important thing and hence any bug will be fixed with atmost > > care. > > > > Also, regarding multi tasksing that you want to achieve. New release of > > bdrb basically encourages cooperative multitasking not preemtpive > > multitasking of threads.I will code some examples for this. > > > > Thanks for your patience and please try the fix i suggested. > > > > > > > > > > > > >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20071206/edfafb6d/attachment-0001.html
On Thu, 2007-12-06 at 16:31 -0800, Andy Tyra wrote:> Hi There, > > Thanks for the help! I have made some progress. > > It looks like the worker is crashing because it is not getting passed > the argument that it requires to run. That would explain why kicking > this worker off manually worked just fine, but doing so in the > scheduler did not. The problem is, I need to be able to pass a > parameter to this worker''s function using the scheduling YML and that > doesn''t seem to be working. Perhaps you can help troubleshoot? > > Here is the relevant method from the worker which is named > ExpireAndRefreshWorker: > > def execute_report(report_name) > myReport = Report.new(report_name) > begin > File.delete (File.join(RAILS_ROOT, ''public'',''reports'', > myReport.const_name + ''.html'')) > rescue > #do nothing > end > renderController = FakeTest.new > renderController.render_report (myReport) > end > > As you can see, it takes one parameter, report_name, which should be a > string. > > Here''s the YML I''m trying to use to schedule it: > > :schedules: > :expire_and_refresh_worker: > :worker_method: execute_report > :data: unfilled_demand > :trigger_args: 0 */15 * * * * * > :job_key: unfilled_demand_scheduleThanks Andy.. with new version you were not able to pass arguments from configuration file and hence the error. But I have fixed this in trunk and your old code should work as usual.> > And here''s the stack trace of the exception that is thrown when the > schedule triggers: (Please note that the line numbers in my stack > trace will not match yours for script/backgroundrb. I''ve added some > additional tasks to this script that add YML scheduling > > rails-root/vendor/plugins/backgroundrb/server/meta_worker.rb:91:in > `execute_report'': wrong number of arguments (0 for 1) (ArgumentError) > from > rails-root/vendor/plugins/backgroundrb/server/meta_worker.rb:91:in > `send'' > from > rails-root/vendor/plugins/backgroundrb/server/meta_worker.rb:91:in > `check_for_timer_events'' > from > rails-root/vendor/plugins/backgroundrb/framework/core.rb:129:in > `start_reactor'' > from > rails-root/vendor/plugins/backgroundrb/framework/core.rb:128:in > `loop'' > from > rails-root/vendor/plugins/backgroundrb/framework/core.rb:128:in > `start_reactor'' > from > rails-root/vendor/plugins/backgroundrb/framework/worker.rb:21:in > `start_worker'' > from > rails-root/vendor/plugins/backgroundrb/framework/packet_master.rb:152:in `fork_and_load'' > from > rails-root/vendor/plugins/backgroundrb/framework/packet_master.rb:111:in `load_workers'' > from > rails-root/vendor/plugins/backgroundrb/framework/packet_master.rb:106:in `each'' > from > rails-root/vendor/plugins/backgroundrb/framework/packet_master.rb:106:in `load_workers'' > from > rails-root/vendor/plugins/backgroundrb/framework/packet_master.rb:19:in `run'' > from > rails-root/vendor/plugins/backgroundrb/server/master_worker.rb:114:in > `initialize'' > from script/backgroundrb:73:in `new'' > from script/backgroundrb:73 > > It appears the parameter specified as :data in the yml is not getting > passed to the function. Is there some other way this should be done? > I don''t see it in the doco. Is it done as :args like in the old > version? > > Also, this is unrelated, but I just thought I''d point it out. The > following stack trace appears in the logs when stopping backgroundrb: > > rails-root/config/../vendor/plugins/backgroundrb/lib/../framework/nbio.rb:20:in `read_data'': Packet::DisconnectError (Packet::DisconnectError) > from > rails-root/vendor/plugins/backgroundrb/framework/worker.rb:47:in > `handle_internal_messages'' > from > rails-root/vendor/plugins/backgroundrb/framework/core.rb:138:in > `start_reactor'' > from > rails-root/vendor/plugins/backgroundrb/framework/core.rb:136:in > `each'' > from > rails-root/vendor/plugins/backgroundrb/framework/core.rb:136:in > `start_reactor'' > from > rails-root/vendor/plugins/backgroundrb/framework/core.rb:128:in `loop'' > from > rails-root/vendor/plugins/backgroundrb/framework/core.rb:128:in > `start_reactor'' > from > rails-root/vendor/plugins/backgroundrb/framework/worker.rb:21:in > `start_worker'' > from > rails-root/vendor/plugins/backgroundrb/framework/packet_master.rb:152:in `fork_and_load'' > from > rails-root/vendor/plugins/backgroundrb/framework/packet_master.rb:128:in `start_worker'' > from > rails-root/vendor/plugins/backgroundrb/server/master_worker.rb:115:in > `initialize'' > from > rails-root/vendor/plugins/backgroundrb/framework/packet_master.rb:18:in `run'' > from > rails-root/vendor/plugins/backgroundrb/server/master_worker.rb:114:in > `initialize'' > from script/backgroundrb:73:in `new'' > from script/backgroundrb:73 > > And, a question. I think I''ve figured out where everything is getting > logged. Will this version eventually log things like the old version? > The old version would report when workers stopped and started, when > schedules were loaded and triggered, etc. Is there somewhere you''d > like this sort of thing submitted as a feature request?Ok, its not a big deal...We can change so as it logs all such things. Thanks for reports.
On Thu, 2007-12-06 at 17:31 -0800, Andy Tyra wrote:> Another issue to report. > > When running the aforementioned worker manually, it runs and seems to > work, but the worker process crashes at the end, leaving this stack > trace: > > rails-root/config/../vendor/plugins/backgroundrb/lib/../framework/nbio.rb:55:in `dump'': can''t dump anonymous class #<Class:0xb6a4ccc8> (TypeError) > from > rails-root/config/../vendor/plugins/backgroundrb/lib/../framework/nbio.rb:55:in `dump_object'' > from > rails-root/vendor/plugins/backgroundrb/framework/worker.rb:32:in > `send_data'' > from > rails-root/vendor/plugins/backgroundrb/server/meta_worker.rb:79:in > `send_response'' > from rails-root/lib/workers/expire_and_refresh_worker.rb:36:in > `process_request'' > from > rails-root/vendor/plugins/backgroundrb/server/meta_worker.rb:48:in > `receive_data'' > from > rails-root/vendor/plugins/backgroundrb/framework/worker.rb:54:in > `receive_internal_data'' > from > rails-root/config/../vendor/plugins/backgroundrb/lib/../framework/bin_parser.rb:30:in `extract'' > from > rails-root/vendor/plugins/backgroundrb/framework/worker.rb:52:in > `receive_internal_data'' > ... 11 levels... > from > rails-root/vendor/plugins/backgroundrb/framework/packet_master.rb:19:in `run'' > from > rails-root/vendor/plugins/backgroundrb/server/master_worker.rb:114:in > `initialize'' > from script/backgroundrb:73:in `new'' > from script/backgroundrb:73Can I have a look at your worker script? If not possible..you can send me personally and I will have a look. I do think, its because of return issue.
Sure, here is the worker code in its entirety. For a little background, this is using the test framework to call a controller function which executes and renders a report. It''s done this way because the previous version of backgroundrb did not support accessing and calling controllers. (If this has changed, that would be awesome, but I''m guessing it has not.) That''s what the extra "FakeTest" class is for. require ''active_support'' require ''action_controller'' require ''action_controller/test_process'' require ''action_view'' class ExpireAndRefreshWorker < BackgrounDRb::MetaWorker set_worker_name :expire_and_refresh_worker def create(args = nil) # this method is called, when worker is loaded for the first time end def execute_report(report_name) myReport = Report.new(report_name) begin File.delete(File.join(RAILS_ROOT, ''public'',''reports'', myReport.const_name + ''.html'')) rescue #do nothing end renderController = FakeTest.new renderController.render_report(myReport) return "I am done." end def process_request(p_data) user_input = p_data[:data] result = self.send(user_input[:worker_method],user_input[:data]) send_response(p_data,result) end end class FakeTest include ActionController::TestProcess def initialize require_dependency ''application'' unless defined?(ApplicationController) @controller = ReportsController.new @request = ActionController::TestRequest.new @response = ActionController::TestResponse.new end def render_report(myReport) get :run, {:id => myReport.const_name} @response end end On Dec 6, 2007 6:24 PM, hemant kumar <gethemant at gmail.com> wrote:> > On Thu, 2007-12-06 at 17:31 -0800, Andy Tyra wrote: > > Another issue to report. > > > > When running the aforementioned worker manually, it runs and seems to > > work, but the worker process crashes at the end, leaving this stack > > trace: > > > > > rails-root/config/../vendor/plugins/backgroundrb/lib/../framework/nbio.rb:55:in > `dump'': can''t dump anonymous class #<Class:0xb6a4ccc8> (TypeError) > > from > > > rails-root/config/../vendor/plugins/backgroundrb/lib/../framework/nbio.rb:55:in > `dump_object'' > > from > > rails-root/vendor/plugins/backgroundrb/framework/worker.rb:32:in > > `send_data'' > > from > > rails-root/vendor/plugins/backgroundrb/server/meta_worker.rb:79:in > > `send_response'' > > from rails-root/lib/workers/expire_and_refresh_worker.rb:36:in > > `process_request'' > > from > > rails-root/vendor/plugins/backgroundrb/server/meta_worker.rb:48:in > > `receive_data'' > > from > > rails-root/vendor/plugins/backgroundrb/framework/worker.rb:54:in > > `receive_internal_data'' > > from > > > rails-root/config/../vendor/plugins/backgroundrb/lib/../framework/bin_parser.rb:30:in > `extract'' > > from > > rails-root/vendor/plugins/backgroundrb/framework/worker.rb:52:in > > `receive_internal_data'' > > ... 11 levels... > > from > > rails-root/vendor/plugins/backgroundrb/framework/packet_master.rb:19:in > `run'' > > from > > rails-root/vendor/plugins/backgroundrb/server/master_worker.rb:114:in > > `initialize'' > > from script/backgroundrb:73:in `new'' > > from script/backgroundrb:73 > > Can I have a look at your worker script? If not possible..you can send > me personally and I will have a look. > > I do think, its because of return issue. > > >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20071206/a06bf598/attachment.html
On Dec 7, 2007 8:45 AM, Andy Tyra <andy.tyra at gmail.com> wrote:> Sure, here is the worker code in its entirety. For a little background, > this is using the test framework to call a controller function which > executes and renders a report. It''s done this way because the previous > version of backgroundrb did not support accessing and calling controllers. > (If this has changed, that would be awesome, but I''m guessing it has not.) > That''s what the extra "FakeTest" class is for. > > require ''active_support'' > require ''action_controller'' > require ''action_controller/test_process'' > require ''action_view'' > > class ExpireAndRefreshWorker < BackgrounDRb::MetaWorker > set_worker_name :expire_and_refresh_worker > def create(args = nil) > # this method is called, when worker is loaded for the first time > end > > > def execute_report(report_name) > myReport = Report.new(report_name) > begin > File.delete(File.join(RAILS_ROOT, ''public'',''reports'', > myReport.const_name + ''.html'')) > rescue > #do nothing > end > renderController = FakeTest.new > renderController.render_report(myReport) > return "I am done." > end > > def process_request(p_data) > user_input = p_data[:data] > result = self.send(user_input[:worker_method],user_input[:data]) > send_response(p_data,result) > end > end > > class FakeTest > > include ActionController::TestProcess > > def initialize > require_dependency ''application'' unless > defined?(ApplicationController) > @controller = ReportsController.new > @request = ActionController::TestRequest.new > @response = ActionController::TestResponse.new > end > > def render_report(myReport) > get :run, {:id => myReport.const_name} > @response > end > end > >Hi Andy, I am trying to reproduce this. Currently since, we do a require ''environment'' all your controllers are available in workers too. However, the error you are reporting seems to have happen when there is communication back and forth between workers or master. But i do not see that case either, since your worker is running in a pretty stand alone manner. What i doubt is, either rails or some other lib that rails is using having a method called "send_response" defined..and thats causing some conflict or something. Let me see it further.
On Fri, 2007-12-07 at 16:09 +0530, hemant wrote:> On Dec 7, 2007 8:45 AM, Andy Tyra <andy.tyra at gmail.com> wrote: > > Sure, here is the worker code in its entirety. For a little background, > > this is using the test framework to call a controller function which > > executes and renders a report. It''s done this way because the previous > > version of backgroundrb did not support accessing and calling controllers. > > (If this has changed, that would be awesome, but I''m guessing it has not.) > > That''s what the extra "FakeTest" class is for. > > > > require ''active_support'' > > require ''action_controller'' > > require ''action_controller/test_process'' > > require ''action_view'' > > > > class ExpireAndRefreshWorker < BackgrounDRb::MetaWorker > > set_worker_name :expire_and_refresh_worker > > def create(args = nil) > > # this method is called, when worker is loaded for the first time > > end > > > > > > def execute_report(report_name) > > myReport = Report.new(report_name) > > begin > > File.delete(File.join(RAILS_ROOT, ''public'',''reports'', > > myReport.const_name + ''.html'')) > > rescue > > #do nothing > > end > > renderController = FakeTest.new > > renderController.render_report(myReport) > > return "I am done." > > end > > > > def process_request(p_data) > > user_input = p_data[:data] > > result = self.send(user_input[:worker_method],user_input[:data]) > > send_response(p_data,result) > > end > > end > > > > class FakeTest > > > > include ActionController::TestProcess > > > > def initialize > > require_dependency ''application'' unless > > defined?(ApplicationController) > > @controller = ReportsController.new > > @request = ActionController::TestRequest.new > > @response = ActionController::TestResponse.new > > end > > > > def render_report(myReport) > > get :run, {:id => myReport.const_name} > > @response > > end > > end > > > >Hi Andy, Sync with latest source code. The bug you encountered should be fixed now. And get rid of older ./script/backgroundrb file and do a setup using: rake backgroundrb:setup Also, now in configuration file you can mention: backgroundrb.yml: :backgroundrb: :port: 11006 :ip: 0.0.0.0 :log: foreground and start backgroundrb in foreground mode using: ./script/backgroundrb this would make sure that all the exceptions appear on stdout. Give it a shot...
I''ll try this over the weekend. Found one bug. Ironically, it occurs if you have NO schedule defined. :) Below is the stack trace and below that is the diff on the file that seems to fix it. rails-root/vendor/plugins/backgroundrb/server/meta_worker.rb:27:in `worker_init'': You have a nil object when you didn''t expect it! (NoMethodError) You might have expected an instance of Array. The error occurred while evaluating nil.[] from rails-root/vendor/plugins/backgroundrb/framework/worker.rb:20:in `start_worker'' from rails-root/vendor/plugins/backgroundrb/framework/packet_master.rb:152:in `fork_and_load'' from rails-root/vendor/plugins/backgroundrb/framework/packet_master.rb:111:in `load_workers'' from rails-root/vendor/plugins/backgroundrb/framework/packet_master.rb:106:in `each'' from rails-root/vendor/plugins/backgroundrb/framework/packet_master.rb:106:in `load_workers'' from rails-root/vendor/plugins/backgroundrb/framework/packet_master.rb:19:in `run'' from rails-root/vendor/plugins/backgroundrb/server/master_worker.rb:116:in `initialize'' from script/backgroundrb:25:in `new'' from script/backgroundrb:25 svn diff meta_worker.rb Index: meta_worker.rb ==================================================================--- meta_worker.rb (revision 227) +++ meta_worker.rb (working copy) @@ -24,7 +24,7 @@ @logger = PacketLogger.new(self) if(@worker_options[:schedule] && no_auto_load) load_schedule_from_args - elsif t_schedule = @config_file[:schedules][worker_name.to_sym] + elsif !@config_file[:schedules].nil? and t_schedule @config_file[:schedules][worker_name.to_sym] @my_schedule = t_schedule @worker_method_arity = self.method (@my_schedule[:worker_method]).arity load_schedule if @my_schedule Thx, Andy On Dec 7, 2007 10:50 PM, hemant kumar <gethemant at gmail.com> wrote:> > On Fri, 2007-12-07 at 16:09 +0530, hemant wrote: > > On Dec 7, 2007 8:45 AM, Andy Tyra <andy.tyra at gmail.com> wrote: > > > Sure, here is the worker code in its entirety. For a little > background, > > > this is using the test framework to call a controller function which > > > executes and renders a report. It''s done this way because the > previous > > > version of backgroundrb did not support accessing and calling > controllers. > > > (If this has changed, that would be awesome, but I''m guessing it has > not.) > > > That''s what the extra "FakeTest" class is for. > > > > > > require ''active_support'' > > > require ''action_controller'' > > > require ''action_controller/test_process'' > > > require ''action_view'' > > > > > > class ExpireAndRefreshWorker < BackgrounDRb::MetaWorker > > > set_worker_name :expire_and_refresh_worker > > > def create(args = nil) > > > # this method is called, when worker is loaded for the first time > > > end > > > > > > > > > def execute_report(report_name) > > > myReport = Report.new(report_name) > > > begin > > > File.delete(File.join(RAILS_ROOT, ''public'',''reports'', > > > myReport.const_name + ''.html'')) > > > rescue > > > #do nothing > > > end > > > renderController = FakeTest.new > > > renderController.render_report(myReport) > > > return "I am done." > > > end > > > > > > def process_request(p_data) > > > user_input = p_data[:data] > > > result = self.send(user_input[:worker_method],user_input[:data]) > > > send_response(p_data,result) > > > end > > > end > > > > > > class FakeTest > > > > > > include ActionController::TestProcess > > > > > > def initialize > > > require_dependency ''application'' unless > > > defined?(ApplicationController) > > > @controller = ReportsController.new > > > @request = ActionController::TestRequest.new > > > @response = ActionController::TestResponse.new > > > end > > > > > > def render_report(myReport) > > > get :run, {:id => myReport.const_name} > > > @response > > > end > > > end > > > > > > > > Hi Andy, > > Sync with latest source code. The bug you encountered should be fixed > now. > > And get rid of older ./script/backgroundrb file and do a setup using: > > rake backgroundrb:setup > > Also, now in configuration > file you can mention: > > backgroundrb.yml: > > :backgroundrb: > :port: 11006 > :ip: 0.0.0.0 > :log: foreground > > and start backgroundrb in foreground mode using: > > ./script/backgroundrb > > this would make sure that all the exceptions appear on stdout. Give it a > shot... > > > >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20071208/120760c8/attachment.html
Looks like this is working! Thanks so much! I''ll try merging this into the app''s mainline tomorrow. One other small bug to report. Backgroundrb doesn''t seem to exit once it has been started and successfully run a worker. I checked for the pid file and was surprised to find it wasn''t there. (This is *WHILE *backgroundrb was running - I never called stop.) Checked the logs and I found the stack below. It seems like backgroundrb is calling "stop" on itself at some point while it is running. Does this make any sense? rails-root/config/../vendor/plugins/backgroundrb/lib/../framework/nbio.rb:20:in `read_data'': Packet::DisconnectError (Packet::DisconnectError) from rails-root/vendor/plugins/backgroundrb/framework/worker.rb:47:in `handle_internal_messages'' from rails-root/vendor/plugins/backgroundrb/framework/core.rb:138:in `start_reactor'' from rails-root/vendor/plugins/backgroundrb/framework/core.rb:136:in `each'' from rails-root/vendor/plugins/backgroundrb/framework/core.rb:136:in `start_reactor'' from rails-root/vendor/plugins/backgroundrb/framework/core.rb:128:in `loop'' from rails-root/vendor/plugins/backgroundrb/framework/core.rb:128:in `start_reactor'' from rails-root/vendor/plugins/backgroundrb/framework/worker.rb:21:in `start_worker'' from rails-root/vendor/plugins/backgroundrb/framework/packet_master.rb:152:in `fork_and_load'' from rails-root/vendor/plugins/backgroundrb/framework/packet_master.rb:128:in `start_worker'' from rails-root/vendor/plugins/backgroundrb/server/master_worker.rb:117:in `initialize'' from rails-root/vendor/plugins/backgroundrb/framework/packet_master.rb:18:in `run'' from rails-root/vendor/plugins/backgroundrb/server/master_worker.rb:116:in `initialize'' from script/backgroundrb:75:in `new'' from script/backgroundrb:75 gins/backgroundrb/server/master_worker.rb:116:in `initialize'' from script/backgroundrb:75:in `new'' from script/backgroundrb:75 On Dec 8, 2007 12:26 AM, Andy Tyra <andy.tyra at gmail.com> wrote:> I''ll try this over the weekend. > > Found one bug. Ironically, it occurs if you have NO schedule defined. > :) Below is the stack trace and below that is the diff on the file that > seems to fix it. > > rails-root/vendor/plugins/backgroundrb/server/meta_worker.rb:27:in > `worker_init'': You have a nil object when you didn''t expect it! > (NoMethodError) > You might have expected an instance of Array. > The error occurred while evaluating nil.[] from > rails-root/vendor/plugins/backgroundrb/framework/worker.rb:20:in > `start_worker'' > from > rails-root/vendor/plugins/backgroundrb/framework/packet_master.rb:152:in > `fork_and_load'' > from > rails-root/vendor/plugins/backgroundrb/framework/packet_master.rb:111:in > `load_workers'' > from > rails-root/vendor/plugins/backgroundrb/framework/packet_master.rb:106:in > `each'' > from > rails-root/vendor/plugins/backgroundrb/framework/packet_master.rb:106:in > `load_workers'' > from > rails-root/vendor/plugins/backgroundrb/framework/packet_master.rb:19:in > `run'' > from > rails-root/vendor/plugins/backgroundrb/server/master_worker.rb:116:in > `initialize'' > from script/backgroundrb:25:in `new'' > from script/backgroundrb:25 > > > svn diff meta_worker.rb > Index: meta_worker.rb > ==================================================================> --- meta_worker.rb (revision 227) > +++ meta_worker.rb (working copy) > @@ -24,7 +24,7 @@ > @logger = PacketLogger.new(self) > if(@worker_options[:schedule] && no_auto_load) > load_schedule_from_args > - elsif t_schedule = @config_file[:schedules][worker_name.to_sym] > + elsif !@config_file[:schedules].nil? and t_schedule > @config_file[:schedules][worker_name.to_sym] > @my_schedule = t_schedule > @worker_method_arity = self.method(@my_schedule[:worker_method]).arity > > load_schedule if @my_schedule > > Thx, > Andy > > > On Dec 7, 2007 10:50 PM, hemant kumar <gethemant at gmail.com> wrote: > > > > > On Fri, 2007-12-07 at 16:09 +0530, hemant wrote: > > > On Dec 7, 2007 8:45 AM, Andy Tyra <andy.tyra at gmail.com> wrote: > > > > Sure, here is the worker code in its entirety. For a little > > background, > > > > this is using the test framework to call a controller function which > > > > executes and renders a report. It''s done this way because the > > previous > > > > version of backgroundrb did not support accessing and calling > > controllers. > > > > (If this has changed, that would be awesome, but I''m guessing it has > > not.) > > > > That''s what the extra "FakeTest" class is for. > > > > > > > > require ''active_support'' > > > > require ''action_controller'' > > > > require ''action_controller/test_process'' > > > > require ''action_view'' > > > > > > > > class ExpireAndRefreshWorker < BackgrounDRb::MetaWorker > > > > set_worker_name :expire_and_refresh_worker > > > > def create(args = nil) > > > > # this method is called, when worker is loaded for the first > > time > > > > end > > > > > > > > > > > > def execute_report(report_name) > > > > myReport = Report.new(report_name) > > > > begin > > > > File.delete(File.join(RAILS_ROOT, > > ''public'',''reports'', > > > > myReport.const_name + ''.html'')) > > > > rescue > > > > #do nothing > > > > end > > > > renderController = FakeTest.new > > > > renderController.render_report(myReport) > > > > return "I am done." > > > > end > > > > > > > > def process_request(p_data) > > > > user_input = p_data[:data] > > > > result = self.send(user_input[:worker_method],user_input[:data]) > > > > send_response(p_data,result) > > > > end > > > > end > > > > > > > > class FakeTest > > > > > > > > include ActionController::TestProcess > > > > > > > > def initialize > > > > require_dependency ''application'' unless > > > > defined?(ApplicationController) > > > > @controller = ReportsController.new > > > > @request = ActionController::TestRequest.new > > > > @response = ActionController::TestResponse.new > > > > end > > > > > > > > def render_report(myReport) > > > > get :run, {:id => myReport.const_name} > > > > @response > > > > end > > > > end > > > > > > > > > > > > Hi Andy, > > > > Sync with latest source code. The bug you encountered should be fixed > > now. > > > > And get rid of older ./script/backgroundrb file and do a setup using: > > > > rake backgroundrb:setup > > > > Also, now in configuration > > file you can mention: > > > > backgroundrb.yml: > > > > :backgroundrb: > > :port: 11006 > > :ip: 0.0.0.0 > > :log: foreground > > > > and start backgroundrb in foreground mode using: > > > > ./script/backgroundrb > > > > this would make sure that all the exceptions appear on stdout. Give it a > > > > shot... > > > > > > > > >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20071208/eade788a/attachment-0001.html