On Tue, Mar 18, 2008 at 9:38 AM, Dave Dupre <gobigdave at gmail.com>
wrote:> I''m having some trouble with using brb for polling for work.
Basically, what
> I want to do is update the state of a record and have brb notice and start
a
> process. I do not want to call ask_work for each process because there may
> be several places where the object''s state is changed, and I only
want to
> process once. Plus, I can schedule things better this way. My first
> thought was to create a worker that polled for records with the correct
> state, but it doesn''t work as expected. Here is what I have with
error
> checking, etc. removed:
>
> class RequestQueuePollerWorker < BackgrounDRb::MetaWorker
> set_worker_name :request_queue_poller_worker
>
> QUEUE_SLEEP_TIME = 30 # seconds
>
> def create(args = nil)
> @running = true
> self.poll_queue
> end
>
> def build_all_matches(args = nil)
> thread_pool.defer(args) do |args|
> requests = Request.find_active(:all)
> requests.each { |request| request.queue! } # using
> acts_as_state_machine
> end
> end
>
> protected
>
> # Was hoping to get multiple threads processing
> def build_matches(args = nil)
> thread_pool.defer(args) do |args|
> request = Request.find(args.to_i)
> request.build_matches
> end
> end
>
> def poll_queue
> Thread.new do
> while @running do
> sleep_time = QUEUE_SLEEP_TIME
> request = nil
> Request.transaction do
> request = Request.find_in_state(:first, :queued, :order =>
> ''requests.updated_at ASC'', :lock => true)
> request.search! unless request.nil? # make sure I
don''t hit
> again
> end
> if request
> self.build_matches(request.id)
> sleep_time = 0
> request = nil
> end
> sleep(sleep_time)
> end
> end
> end
>
> def stop_polling
> @running = false
> end
>
> end
>
> In my test case, there are 20 Request items. Normally, it would take about
5
> seconds to call Request.build_matches on 20 Request items. However, asking
> for work like so:
>
> MiddleMan.ask_work(:worker => :request_queue_poller_worker,
:worker_method
> => :build_all_matches)
>
> It takes 20 minutes(!) to process the same 20 Request items. Processing
> basically c r a w l s along. I expected to immediately see all the states
> change to :queued, and then very quickly changed to :searching. However, it
> doesn''t happen that way. Things go to queued pretty quickly, by
searching
> takes FOREVER. It''s like poll_queue and build_matches
don''t get any CPU
> cycles.
>
> Is there a recommended way to do something like this?
>
> Please note, I do not want to use a periodic timer because I don''t
know how
> long handling a message will take. Plus, if there are more items in the
> queue, I don''t want to wait around to get them. My thought is to
generalize
> this as a db queue eventually.
>
You should never put sleep anywhere in your worker code. Mainly
because all the workers are reactive in nature and putting sleep
anywhere may block main reactor loop. If you want certain action to be
executed like a polling method, you should investigate next_turn
method, which will invoke specified block on each turn of reactor loop
inside worker.
Remove the sleep and look into next_turn and see how are your results.
Also, sorry for my late reply, since I am stuck in a village almost
without internet and electricity.
--
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.
http://gnufied.org