Backgroundrb devel - Jun 2006 - Cannot invoke ActiveRecord subclass accessors in DRb worker

Hi there,

I''m trying to create a worker to do some asynchronous web client work
and update the data model with its results. It seems, however, that
every call I make to an ActiveRecord subclass causes the whole thing
to block. At least I think it''s blocked. Right now I''m just
using
debug log output.

Is there a decent way to debug/test workers I can try?

Is there any insight as to why this might be happening (thread safety
issues, etc)?

Cheers

Dorian

On Jun 15, 2006, at 5:42 AM, dorian taylor wrote:
> Hi there,
>
> I''m trying to create a worker to do some asynchronous web client
work
> and update the data model with its results. It seems, however, that
> every call I make to an ActiveRecord subclass causes the whole thing
> to block. At least I think it''s blocked. Right now I''m
just using
> debug log output.
>
> Is there a decent way to debug/test workers I can try?
>
> Is there any insight as to why this might be happening (thread safety
> issues, etc)?
>
> Cheers
>
> Dorian
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>
Dorian-

	Can you show me an example of your worker? You can shave it down to  
just the problem area. I will try to reproduce the problem and get  
back to you with an answer.

	But yeah threads can be hard to debug. But each worker does run in  
it''s own thread and I have tested workers that use activerecord  
models by starting a bunch of them at once and they did not block and  
performed as they should. But I am interested to find any problems  
and fix them so please give a concrete example.

Thanks
-Ezra

argh, forgot to add recipient.

---------- Forwarded message ----------
From: dorian taylor <dorian.taylor.lists at gmail.com>
Date: Jun 15, 2006 1:02 PM
Subject: Re: [Backgroundrb-devel] Cannot invoke ActiveRecord subclass
accessors in DRb worker
To: Ezra Zygmuntowicz <ezmobius at gmail.com>


On 6/15/06, Ezra Zygmuntowicz <ezmobius at gmail.com>
wrote:
> Dorian-
>
>         Can you show me an example of your worker? You can shave it down to
> just the problem area. I will try to reproduce the problem and get
> back to you with an answer.
Sure, it''s as simple as this:

class FoobarWorker
    include DRbUndumped

    def initialize(options={})
        @progress = 0
        @logger = BACKGROUNDRB_LOGGER
        start_working
    end

    def start_working(args={})
        Thread.new do
            @logger.debug "Trying to invoke stuff"
            person = Person.find(2)
            @logger.debug "First name is #{person.firstname}"
        end
    end

end

Or the new version:

class FoobarWorker2 < BackgrounDRb::Rails
  def do_work (args)
    @logger.debug("Attempting to get ActiveRecord instance")
    person = Person.find(2)
    @logger.debug("First name is #{person.firstname}")
  end
end

In both cases, execution stops after (during?) the attempt to find()
the record. I set the ulimit and no core is being dumped. The Rails is
from trunk, and the database is MySQL 5.0 on a Debian/testing system.
>         But yeah threads can be hard to debug. But each worker does run in
> it''s own thread and I have tested workers that use activerecord
> models by starting a bunch of them at once and they did not block and
> performed as they should. But I am interested to find any problems
> and fix them so please give a concrete example.
I''m sure you did test workers with ActiveRecord as I imagine it would
be a pretty common thing to want to do.

Perhaps it''s something with the underlying software then? I''m
not
sure. The ActiveRecord instances work fine if I invoke them directly
within a Thread in script/console.

Any insight would be great.

Cheers

Dorian

.
>
> I''m sure you did test workers with ActiveRecord as I imagine it
would
> be a pretty common thing to want to do.
>
> Perhaps it''s something with the underlying software then?
I''m not
> sure. The ActiveRecord instances work fine if I invoke them directly
> within a Thread in script/console.
>
> Any insight would be great.
>
> Cheers
>
> Dorian
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>

Hey Dorian-


	I''m not sure what could be the problem. I just made a test worker  
that creates, reads and writes ActiveRecords in one worker in a loop  
and ran 10 of them at once and it worked for me. Here is the worker:

class TestWorker < BackgrounDRb::Rails

   def do_work(args)
     10.times do |i|
       post = Post.create :title => "New: #{args[:id]}:#{i}"
       @logger.debug "#{args[:id]} : #{self.object_id}: " +
(post.title)
       post = Post.find(args[:id])
       post.title = "New title from worker #{args[:id]}: time: #{i}"
       post.save
       @logger.debug post.title
     end
   end

end

THen I called them like this:

   def index
     MiddleMan.new_worker :class => :test_worker, :args => {:id => 1}
     MiddleMan.new_worker :class => :test_worker, :args => {:id => 2}
     MiddleMan.new_worker :class => :test_worker, :args => {:id => 3}
     MiddleMan.new_worker :class => :test_worker, :args => {:id => 4}
     MiddleMan.new_worker :class => :test_worker, :args => {:id => 5}
     MiddleMan.new_worker :class => :test_worker, :args => {:id => 6}
     MiddleMan.new_worker :class => :test_worker, :args => {:id => 7}
     MiddleMan.new_worker :class => :test_worker, :args => {:id => 8}
     MiddleMan.new_worker :class => :test_worker, :args => {:id => 9}
     MiddleMan.new_worker :class => :test_worker, :args => {:id => 10}
   end

	And here is a little except from the log so you can see that it is  
working:

1 : 10643990: New: 1:0
New title from worker 1: time: 0
2 : 10624220: New: 2:0
New title from worker 2: time: 0
1 : 10643990: New: 1:1
3 : 10599690: New: 3:0
2 : 10624220: New: 2:1
New title from worker 1: time: 1
4 : 10574310: New: 4:0
New title from worker 3: time: 0
New title from worker 2: time: 1
1 : 10643990: New: 1:2
5 : 10549280: New: 5:0
New title from worker 4: time: 0
3 : 10599690: New: 3:1
2 : 10624220: New: 2:2
New title from worker 1: time: 2
6 : 11499220: New: 6:0
New title from worker 5: time: 0
4 : 10574310: New: 4:1
7 : 11472730: New: 7:0
New title from worker 3: time: 1
New title from worker 2: time: 2
1 : 10643990: New: 1:3
8 : 11435250: New: 8:0
New title from worker 6: time: 0
5 : 10549280: New: 5:1
New title from worker 4: time: 1
9 : 11396760: New: 9:0
New title from worker 7: time: 0
3 : 10599690: New: 3:2


	I''m not sure what the problem could be in your end. If you want I  
can send you a copy of the sample app I have where this works but I  
think it should be working for you. You have the latest version of  
the plugin? And have you tried stoping and restarting the drb server  
and the webserver? I''d like to help you get this working so let me kn  
ow if you can''t get around it.

Cheers-
-Ezra

The problem was that we were using ERB in the config/database.yml
which wasn''t getting evaluated and therefore the database
wouldn''t
connect. This trivial patch will fix that.

Dorian
-------------- next part --------------
A non-text attachment was scrubbed...
Name: backgroundrb.diff
Type: application/octet-stream
Size: 1633 bytes
Desc: not available
Url :
http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20060616/f8795ed9/attachment.obj

On Jun 16, 2006, at 2:20 AM, dorian taylor wrote:
> The problem was that we were using ERB in the config/database.yml
> which wasn''t getting evaluated and therefore the database
wouldn''t
> connect. This trivial patch will fix that.
>
> Dorian
> <backgroundrb.diff>
> _______________________________________________

	Ahh, thanks Dorian. I hadn''t even thought of that since I
don''t tend
to use erb in my yml but I will add the patch as soon as i get a chance.

Thanks!
-Ezra