We''re currently using a really old version of BackgrounDRb in a small production environment and we''re thinking about upgrading to 2.0, primarily because we want to push it a little harder and have it manage a few more things. Problem is, we''re a little confused on some changes that have been made to the new version. Perhaps these are stupid questions, or perhaps they have been answered elsewhere, but we can''t find them in the mailing list archive, and the current documentation doesn''t seem to address them, so here goes: 1. Auto-generated job keys. In our current version we can do something like @job_key = MiddleMan.new_worker(:class => :test_worker, :args => {"some args go here"}) which will return back a automatically generated random job key that we can then use to access the worker at a later stage. 2.0 doesn''t seem to do this, but instead requires us to generate a job key for it - is this right or are we just missing something? 2. Accessing a particular working In our version we can access a particular worker by doing something like: my_worker = MiddleMan.get_worker(@job_key) which, of course, returns our worker. There doesn''t seem to be a similar function for 2.0. We''ve seen the example for getting all the workers, but not an example for getting one particular worker. Is this possible? 3. Accessing worker attributes We have some workers that are defined something like this: class TestWorker < BackgrounDRb::Rails attr_reader :progress attr_reader :total attr_reader :status def do_work(args = nil) @status = "started" @total = some_object.length #blah blah blah... @status = "working" @progress += 1 #blah blah blah... @status = "complete" end end For long running tasks we''d grab the worker object similar to the example given in 2 above and then query it''s parameters like so puts my_worker.status puts my_worker.length while(my_worker.progress < 100) { puts my_worker.progress } Now, we realise that you can use something like MiddleMan.ask_status(:worker => :foo_worker) but it seems that it''s possible to only have one attribute per worker. Is it possible to have multiple attributes per worker and if so how do we access them? We''ve actually spent some time trying out a few of these things but we''ve had very little success. I suspect that the new 2.0 system is so different from the version that we''re using that we''re just having a difficult time getting our heads around it. Any help here would be greatly appreciated. Dale -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080119/077ce5a8/attachment.html
Hi, On Jan 20, 2008 7:50 AM, Dale Cook <dale at dalecook.us> wrote:> We''re currently using a really old version of BackgrounDRb in a small > production environment and we''re thinking about upgrading to 2.0, primarily > because we want to push it a little harder and have it manage a few more > things. Problem is, we''re a little confused on some changes that have been > made to the new version. Perhaps these are stupid questions, or perhaps they > have been answered elsewhere, but we can''t find them in the mailing list > archive, and the current documentation doesn''t seem to address them, so here > goes: > > 1. Auto-generated job keys. > In our current version we can do something like > > @job_key = MiddleMan.new_worker(:class => :test_worker, :args => {"some args > go here"}) > > which will return back a automatically generated random job key that we can > then use to access the worker at a later stage. 2.0 doesn''t seem to do this, > but instead requires us to generate a job key for it - is this right or are > we just missing something?Yes thats right, you will have to specify your job_key while calling the worker. Even in older versions, you had the option to supply the job_key.> > 2. Accessing a particular working > In our version we can access a particular worker by doing something like: > > my_worker = MiddleMan.get_worker(@job_key) > > which, of course, returns our worker. There doesn''t seem to be a similar > function for 2.0. We''ve seen the example for getting all the workers, but > not an example for getting one particular worker. Is this possible?There is a major difference that in older version you had access to worker environment through DRb. For stability sake, there are no threads and hence you don''t have direct access to worker environment. If you want to execute a task, you have to use: MiddleMan.ask_work(:worker => :foo_worker,:job_key => @job_key)> > 3. Accessing worker attributes > We have some workers that are defined something like this: > > class TestWorker < BackgrounDRb::Rails > > attr_reader :progress > attr_reader :total > attr_reader :status > > def do_work(args = nil) > > @status = "started" > @total = some_object.length > > #blah blah blah... > > @status = "working" > @progress += 1 > > #blah blah blah... > > @status = "complete" > > end > > end > > For long running tasks we''d grab the worker object similar to the example > given in 2 above and then query it''s parameters like so > > puts my_worker.status > puts my_worker.length > > while(my_worker.progress < 100) > { > puts my_worker.progress > } > > Now, we realise that you can use something like MiddleMan.ask_status(:worker > => :foo_worker) but it seems that it''s possible to only have one attribute > per worker. Is it possible to have multiple attributes per worker and if so > how do we access them? > > We''ve actually spent some time trying out a few of these things but we''ve > had very little success. I suspect that the new 2.0 system is so different > from the version that we''re using that we''re just having a difficult time > getting our heads around it. Any help here would be greatly appreciated.Again remember you don''t have direct access to worker environment through MiddleMan. Its probably good and bad. Good because it reduces complexity and bad because it sacrifices on flexibility. However, you can easily compile your attributes in a hash and access it from rails. For example: class HelloWorker < BackgrounDRb::MetaWorker set_worker_name :hello_worker def create(args = nil) @worker_attributes = {:length => 0, :percentage => 0} register_status(@worker_status) end def start_work loop do # do some processing end @worker_attributes[:length] = 100 @worker_attributes[:percentage] = 100 register_status(@worker_attributes) end end Issue of not able to access worker attributes, is it a blocker for you? -- Let them talk of their oriental summer climes of everlasting conservatories; give me the privilege of making my own summer with my own coals. http://gnufied.org
Thanks for the reply. It seems that there''s certainly a fundamental difference between our version and 2.0 - I think this has been causing our confusion. None of this is a show stopper for us, but let me see if I''ve got this straight, let''s say we had a worker that took a fairly long running task (lets say it pulls down csv data from another server and processes it) and was capable of updating a number of attributes so that we could report back to the user that status and progress. Would it look something like this? The worker code: class CsvWorker < BackgrounDRb::MetaWorker set_worker_name :csv_worker def create(args = nil) @worker_attributes = {:status => 0, total => 0, :progress => 0} register_status(@worker_status) end def process_file(args = nil) #these values are necessary to login to the external csv server login = args[:login] password = args[:password] server_location = args[:server] @worker_attributes[:total] = 0 @worker_attributes[:progress] = 0 @worker_attributes[:status] = "contacting server" register_status(@worker_attributes) # not shown here: login to the server, download the file, load the lines of the csv file into an array, etc.... @worker_attributes[:total] = csv_array.length @worker_attributes[:status] = "processing" register_status(@worker_attributes) for record in csv_array #process the record, or whatever.... @worker_attributes[:progress] += 1 register_status(@worker_attributes) end end end Then our controller code would look something like: class ProcessesController < ApplicationController def import #generate_job_key generates a new random key for us @job_key = generate_job_key MiddleMan.new_worker(:worker => :csv_worker, :job_key => @job_key) MiddleMan.ask_work(:worker => :csv_worker, :job_key => @job_key, :worker_method => :process_file, :data => {:login => "mylogin", :password => "mypassword", :server => "192.168.22.55"}) #send an Ajax.Request function back tot he page to make it access the status method render :update do |page| page << remote_function(:url => {:action => :status, :controller => "processes", :id => @job_key}) end end def status results = MiddleMan.ask_status(:worker => :csv_worker, :job_key params[:id]) #update the page with relevent progress information render :update do |page| if(results[:status] = "processing" ) page[''report-progress''].replace_html "#{results[:progress]} records processed out of a total of #{results[:total]" page << remote_function(:url => {:action => :status, :controller => "processes", :id => params[:id]}) else page[''report-progress''].replace_html results[:status] end end end end I won''t include the actual view since it''s just a simple page with a "start" link and a div that provides the update information. We''ve already tried something like this (actually this is a simplification of the test code we''re using) but we still haven''t had much success - perhaps there''s simply something wrong here. Anyway, I also wanted to say thanks to everyone who''s worked on BackgrounDRb, even if the new version doesn''t work out for us (I suspect it will) it''s an amazing project and we''re really thankful that people have taken the time and energy to develop it. Dale On Jan 19, 2008 7:44 PM, hemant <gethemant at gmail.com> wrote:> Hi, > > On Jan 20, 2008 7:50 AM, Dale Cook <dale at dalecook.us> wrote: > > We''re currently using a really old version of BackgrounDRb in a small > > production environment and we''re thinking about upgrading to 2.0, > primarily > > because we want to push it a little harder and have it manage a few more > > things. Problem is, we''re a little confused on some changes that have > been > > made to the new version. Perhaps these are stupid questions, or perhaps > they > > have been answered elsewhere, but we can''t find them in the mailing list > > archive, and the current documentation doesn''t seem to address them, so > here > > goes: > > > > 1. Auto-generated job keys. > > In our current version we can do something like > > > > @job_key = MiddleMan.new_worker(:class => :test_worker, :args => {"some > args > > go here"}) > > > > which will return back a automatically generated random job key that we > can > > then use to access the worker at a later stage. 2.0 doesn''t seem to do > this, > > but instead requires us to generate a job key for it - is this right or > are > > we just missing something? > > Yes thats right, you will have to specify your job_key while calling the > worker. > Even in older versions, you had the option to supply the job_key. > > > > > 2. Accessing a particular working > > In our version we can access a particular worker by doing something > like: > > > > my_worker = MiddleMan.get_worker(@job_key) > > > > which, of course, returns our worker. There doesn''t seem to be a similar > > function for 2.0. We''ve seen the example for getting all the workers, > but > > not an example for getting one particular worker. Is this possible? > > There is a major difference that in older version you had access to > worker environment through DRb. > For stability sake, there are no threads and hence you don''t have > direct access to worker environment. > > If you want to execute a task, you have to use: > > MiddleMan.ask_work(:worker => :foo_worker,:job_key => @job_key) > > > > > 3. Accessing worker attributes > > We have some workers that are defined something like this: > > > > class TestWorker < BackgrounDRb::Rails > > > > attr_reader :progress > > attr_reader :total > > attr_reader :status > > > > def do_work(args = nil) > > > > @status = "started" > > @total = some_object.length > > > > #blah blah blah... > > > > @status = "working" > > @progress += 1 > > > > #blah blah blah... > > > > @status = "complete" > > > > end > > > > end > > > > For long running tasks we''d grab the worker object similar to the > example > > given in 2 above and then query it''s parameters like so > > > > puts my_worker.status > > puts my_worker.length > > > > while(my_worker.progress < 100) > > { > > puts my_worker.progress > > } > > > > Now, we realise that you can use something like MiddleMan.ask_status > (:worker > > => :foo_worker) but it seems that it''s possible to only have one > attribute > > per worker. Is it possible to have multiple attributes per worker and if > so > > how do we access them? > > > > We''ve actually spent some time trying out a few of these things but > we''ve > > had very little success. I suspect that the new 2.0 system is so > different > > from the version that we''re using that we''re just having a difficult > time > > getting our heads around it. Any help here would be greatly appreciated. > > Again remember you don''t have direct access to worker environment > through MiddleMan. Its probably good and bad. Good because it reduces > complexity and bad because it sacrifices on flexibility. However, you > can easily compile your attributes in a hash and access it from rails. > For example: > > class HelloWorker < BackgrounDRb::MetaWorker > set_worker_name :hello_worker > def create(args = nil) > @worker_attributes = {:length => 0, :percentage => 0} > register_status(@worker_status) > end > > def start_work > loop do > # do some processing > end > @worker_attributes[:length] = 100 > @worker_attributes[:percentage] = 100 > register_status(@worker_attributes) > end > end > > Issue of not able to access worker attributes, is it a blocker for you? > > > -- > Let them talk of their oriental summer climes of everlasting > conservatories; give me the privilege of making my own summer with my > own coals. > > http://gnufied.org > _______________________________________________ > Backgroundrb-devel mailing list > Backgroundrb-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/backgroundrb-devel >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080120/5370ec97/attachment-0001.html
Hi Dale, On Jan 21, 2008 3:20 AM, Dale Cook <dale at dalecook.us> wrote:> Thanks for the reply. It seems that there''s certainly a fundamental > difference between our version and 2.0 - I think this has been causing our > confusion. > > None of this is a show stopper for us, but let me see if I''ve got this > straight, let''s say we had a worker that took a fairly long running task > (lets say it pulls down csv data from another server and processes it) and > was capable of updating a number of attributes so that we could report back > to the user that status and progress. Would it look something like this? > > The worker code: > > class CsvWorker < BackgrounDRb::MetaWorker > > set_worker_name :csv_worker > > def create(args = nil) > @worker_attributes = {:status => 0, total => 0, :progress => 0} > > register_status(@worker_status) > end > > def process_file(args = nil) > > #these values are necessary to login to the external csv server > login = args[:login] > password = args[:password] > server_location = args[:server] > > @worker_attributes[:total] = 0 > @worker_attributes[:progress] = 0 > @worker_attributes[:status] = "contacting server" > register_status(@worker_attributes) > > # not shown here: login to the server, download the file, load the lines > of the csv file into an array, etc.... > > @worker_attributes[:total] = csv_array.length > @worker_attributes[:status] = "processing" > register_status(@worker_attributes) > > for record in csv_array > > #process the record, or whatever.... > > @worker_attributes[:progress] += 1 > > register_status(@worker_attributes) > > end > > end > > end > > Then our controller code would look something like: > > class ProcessesController < ApplicationController > > def import > > #generate_job_key generates a new random key for us > @job_key = generate_job_key > > MiddleMan.new_worker(:worker => :csv_worker, :job_key => @job_key) > > MiddleMan.ask_work(:worker => :csv_worker, :job_key => @job_key, > :worker_method => :process_file, :data => {:login => "mylogin", :password => > "mypassword", :server => " 192.168.22.55"}) > > #send an Ajax.Request function back tot he page to make it access the > status method > render :update do |page| > > page << remote_function(:url => {:action => :status, :controller => > "processes", :id => @job_key}) > > end > > end > > def status > > results = MiddleMan.ask_status(:worker => :csv_worker, :job_key > params[:id]) > > #update the page with relevent progress information > render :update do |page| > > if(results[:status] = "processing" ) > > page[''report-progress''].replace_html "#{results[:progress]} records > processed out of a total of #{results[:total]" > > page << remote_function(:url => {:action => :status, :controller => > "processes", :id => params[:id]}) > > else > > page[''report-progress''].replace_html results[:status] > > end > > end > > end > > end > > I won''t include the actual view since it''s just a simple page with a "start" > link and a div that provides the update information. > > We''ve already tried something like this (actually this is a simplification > of the test code we''re using) but we still haven''t had much success - > perhaps there''s simply something wrong here. > > Anyway, I also wanted to say thanks to everyone who''s worked on > BackgrounDRb, even if the new version doesn''t work out for us (I suspect it > will) it''s an amazing project and we''re really thankful that people have > taken the time and energy to develop it. >Yes your code looks alright, at least the part thats dealing with BackgrounDRb. So whats the error you are getting? If you tell us about the problem, we can surely straighten it out. Which part is not working? Do you have errors in RAILS_ROOT/log/backgroundrb_debug.log ? You can turn log messages in foreground and find out where its failing and stuff like that.
Actually, thanks to your guidance, we played around with it the morning and it''s now working like a charm. I really think having a better understanding of how it works now really helped - thanks a bunch. Dale On Jan 20, 2008 2:14 PM, hemant <gethemant at gmail.com> wrote:> Hi Dale, > > On Jan 21, 2008 3:20 AM, Dale Cook <dale at dalecook.us> wrote: > > Thanks for the reply. It seems that there''s certainly a fundamental > > difference between our version and 2.0 - I think this has been causing > our > > confusion. > > > > None of this is a show stopper for us, but let me see if I''ve got this > > straight, let''s say we had a worker that took a fairly long running task > > (lets say it pulls down csv data from another server and processes it) > and > > was capable of updating a number of attributes so that we could report > back > > to the user that status and progress. Would it look something like this? > > > > The worker code: > > > > class CsvWorker < BackgrounDRb::MetaWorker > > > > set_worker_name :csv_worker > > > > def create(args = nil) > > @worker_attributes = {:status => 0, total => 0, :progress => 0} > > > > register_status(@worker_status) > > end > > > > def process_file(args = nil) > > > > #these values are necessary to login to the external csv server > > login = args[:login] > > password = args[:password] > > server_location = args[:server] > > > > @worker_attributes[:total] = 0 > > @worker_attributes[:progress] = 0 > > @worker_attributes[:status] = "contacting server" > > register_status(@worker_attributes) > > > > # not shown here: login to the server, download the file, load the > lines > > of the csv file into an array, etc.... > > > > @worker_attributes[:total] = csv_array.length > > @worker_attributes[:status] = "processing" > > register_status(@worker_attributes) > > > > for record in csv_array > > > > #process the record, or whatever.... > > > > @worker_attributes[:progress] += 1 > > > > register_status(@worker_attributes) > > > > end > > > > end > > > > end > > > > Then our controller code would look something like: > > > > class ProcessesController < ApplicationController > > > > def import > > > > #generate_job_key generates a new random key for us > > @job_key = generate_job_key > > > > MiddleMan.new_worker(:worker => :csv_worker, :job_key => @job_key) > > > > MiddleMan.ask_work(:worker => :csv_worker, :job_key => @job_key, > > :worker_method => :process_file, :data => {:login => "mylogin", > :password => > > "mypassword", :server => " 192.168.22.55"}) > > > > #send an Ajax.Request function back tot he page to make it access > the > > status method > > render :update do |page| > > > > page << remote_function(:url => {:action => :status, :controller > => > > "processes", :id => @job_key}) > > > > end > > > > end > > > > def status > > > > results = MiddleMan.ask_status(:worker => :csv_worker, :job_key > > params[:id]) > > > > #update the page with relevent progress information > > render :update do |page| > > > > if(results[:status] = "processing" ) > > > > page[''report-progress''].replace_html "#{results[:progress]} > records > > processed out of a total of #{results[:total]" > > > > page << remote_function(:url => {:action => :status, :controller > => > > "processes", :id => params[:id]}) > > > > else > > > > page[''report-progress''].replace_html results[:status] > > > > end > > > > end > > > > end > > > > end > > > > I won''t include the actual view since it''s just a simple page with a > "start" > > link and a div that provides the update information. > > > > We''ve already tried something like this (actually this is a > simplification > > of the test code we''re using) but we still haven''t had much success - > > perhaps there''s simply something wrong here. > > > > Anyway, I also wanted to say thanks to everyone who''s worked on > > BackgrounDRb, even if the new version doesn''t work out for us (I suspect > it > > will) it''s an amazing project and we''re really thankful that people have > > taken the time and energy to develop it. > > > > Yes your code looks alright, at least the part thats dealing with > BackgrounDRb. So whats the error you are getting? If you tell us about > the problem, we can surely straighten it out. Which part is not > working? Do you have errors in RAILS_ROOT/log/backgroundrb_debug.log ? > You can turn log messages in foreground and find out where its failing > and stuff like that. > _______________________________________________ > Backgroundrb-devel mailing list > Backgroundrb-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/backgroundrb-devel >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080121/8b353a29/attachment.html