I have a question concerning where to use the synchronize function. The point of it is to put all of the code blacks passed to this function in a queue. This make sure that code gets ran one at a time. This is required when doing any kind of database activity in backgrondrb and multiple threads. Otherwise I lose the connection to my database. So here is my question: Do I need to do: @lock = Mutext.new And do this for EVERY database interaction WITHIN background rb: @lock.synchronize do #some db activity, like model.save!, model.find, anything that queries the db end Or do I need to wrap ALL database activity in a sychronize method? Including various database activity in my controllers that will never be called by backgroundrb. Because what if someone makes a request to somedomain.com/products/create while my backgroundrb script is calling Product.find_all at the same time? Even though they aren''t both in the same process would this cause a problem? Or is synchronize ONLY meant for using in a single process environment where other processes are irrelevant? Sorry if this is confusing, any help is greatly appreciated. -- Posted via http://www.ruby-forum.com/.
On Jul 6, 2006, at 3:42 PM, Ben Johnson wrote:> I have a question concerning where to use the synchronize function. > The > point of it is to put all of the code blacks passed to this > function in > a queue. This make sure that code gets ran one at a time. This is > required when doing any kind of database activity in backgrondrb and > multiple threads. Otherwise I lose the connection to my database. > > So here is my question: > > Do I need to do: > @lock = Mutext.new > > And do this for EVERY database interaction WITHIN background rb: > @lock.synchronize do > #some db activity, like model.save!, model.find, anything that > queries the db > end > > Or do I need to wrap ALL database activity in a sychronize method? > Including various database activity in my controllers that will > never be > called by backgroundrb. > > Because what if someone makes a request to > somedomain.com/products/create while my backgroundrb script is calling > Product.find_all at the same time? Even though they aren''t both in the > same process would this cause a problem? Or is synchronize ONLY meant > for using in a single process environment where other processes are > irrelevant? > > Sorry if this is confusing, any help is greatly appreciated.Hey Ben- You have it right. You do not need to use a mutex in rails at all since the drb server is a separate process with its own db connection. So don''t worry about it on the rails side of things. I am still experimenting with different techniques for allowing concurrency with active record so you won''t need a mutex. Are you using the latest version of the plugin? I added this line recently: ActiveRecord::Base.allow_concurrency = true This is supposed to allow for thread safe access to AR objects. BUt I have had mixed results. Maybe you can send me a copy of your worker class and I can help you get it working. Also consider joining the backgroundrb mailing list: *Mailing list: http://rubyforge.org/mailman/listinfo/backgroundrb-devel Cheers- -Ezra -------------- next part -------------- An HTML attachment was scrubbed... URL: http://wrath.rubyonrails.org/pipermail/rails/attachments/20060706/18dae8f2/attachment-0001.html
Where exactly would I add this line? ActiveRecord::Base.allow_concurrency = true Thanks a lot for the help. Ezra Zygmuntowicz wrote:> On Jul 6, 2006, at 3:42 PM, Ben Johnson wrote: > >> Do I need to do: >> never be >> called by backgroundrb. >> >> Because what if someone makes a request to >> somedomain.com/products/create while my backgroundrb script is calling >> Product.find_all at the same time? Even though they aren''t both in the >> same process would this cause a problem? Or is synchronize ONLY meant >> for using in a single process environment where other processes are >> irrelevant? >> >> Sorry if this is confusing, any help is greatly appreciated. > > > Hey Ben- > > You have it right. You do not need to use a mutex in rails at all > since the drb server is a separate process with its own db > connection. So don''t worry about it on the rails side of things. I am > still experimenting with different techniques for allowing > concurrency with active record so you won''t need a mutex. Are you > using the latest version of the plugin? I added this line recently: > > ActiveRecord::Base.allow_concurrency = true > > This is supposed to allow for thread safe access to AR objects. BUt > I have had mixed results. Maybe you can send me a copy of your worker > class and I can help you get it working. Also consider joining the > backgroundrb mailing list: > > *Mailing list: http://rubyforge.org/mailman/listinfo/backgroundrb-devel > > > Cheers- > -Ezra-- Posted via http://www.ruby-forum.com/.
Ben-
Take a look in the script/backgroundrb/start file. You should see
that line in there like this:
if CONFIG[''load_rails'']
ActiveRecord::Base.allow_concurrency = true
ActiveRecord::Base.establish_connection(YAML.load(ERB.new(IO.read("#
{RAILS_ROOT}/#{CONFIG[''database_yml'']}")).result)[CONFIG
[''environment'']])
end
So if you don''t see that line in the start script then you need to
add it or upgrade your version of the plugin.
Cheers-
-Ezra
On Jul 7, 2006, at 10:15 AM, Ben Johnson wrote:
> Where exactly would I add this line?
>
> ActiveRecord::Base.allow_concurrency = true
>
> Thanks a lot for the help.
>
> Ezra Zygmuntowicz wrote:
>> On Jul 6, 2006, at 3:42 PM, Ben Johnson wrote:
>>
>>> Do I need to do:
>>> never be
>>> called by backgroundrb.
>>>
>>> Because what if someone makes a request to
>>> somedomain.com/products/create while my backgroundrb script is
>>> calling
>>> Product.find_all at the same time? Even though they aren''t
both
>>> in the
>>> same process would this cause a problem? Or is synchronize ONLY
>>> meant
>>> for using in a single process environment where other processes are
>>> irrelevant?
>>>
>>> Sorry if this is confusing, any help is greatly appreciated.
>>
>>
>> Hey Ben-
>>
>> You have it right. You do not need to use a mutex in rails at all
>> since the drb server is a separate process with its own db
>> connection. So don''t worry about it on the rails side of
things. I am
>> still experimenting with different techniques for allowing
>> concurrency with active record so you won''t need a mutex. Are
you
>> using the latest version of the plugin? I added this line recently:
>>
>> ActiveRecord::Base.allow_concurrency = true
>>
>> This is supposed to allow for thread safe access to AR objects. BUt
>> I have had mixed results. Maybe you can send me a copy of your worker
>> class and I can help you get it working. Also consider joining the
>> backgroundrb mailing list:
>>
>> *Mailing list: http://rubyforge.org/mailman/listinfo/backgroundrb-
>> devel
>>
>>
>> Cheers-
>> -Ezra
>
>
> --
> Posted via http://www.ruby-forum.com/.
> _______________________________________________
> Rails mailing list
> Rails@lists.rubyonrails.org
> http://lists.rubyonrails.org/mailman/listinfo/rails
Also,
I have the following method in my worker class. I ran that in 6
different threads at once. No allow_concurrency and no mutex.synchronize
and I didn''t have a problem. They all ran over night for about 8 hours
without a problem. The updated_at field was changing every second.
Shouldn''t it lose connection to my database is 6 different threads are
accessing the database as fast as they can?
def monitor_event(event_id)
event = Event.find event_id
@event_monitor_threads[event.id] = Thread.new(event) do |event|
while Event.exists? event.id
event.save!
end
end
end
Ezra Zygmuntowicz wrote:> On Jul 6, 2006, at 3:42 PM, Ben Johnson wrote:
>
>> Do I need to do:
>> never be
>> called by backgroundrb.
>>
>> Because what if someone makes a request to
>> somedomain.com/products/create while my backgroundrb script is calling
>> Product.find_all at the same time? Even though they aren''t
both in the
>> same process would this cause a problem? Or is synchronize ONLY meant
>> for using in a single process environment where other processes are
>> irrelevant?
>>
>> Sorry if this is confusing, any help is greatly appreciated.
>
>
> Hey Ben-
>
> You have it right. You do not need to use a mutex in rails at all
> since the drb server is a separate process with its own db
> connection. So don''t worry about it on the rails side of things. I
am
> still experimenting with different techniques for allowing
> concurrency with active record so you won''t need a mutex. Are you
> using the latest version of the plugin? I added this line recently:
>
> ActiveRecord::Base.allow_concurrency = true
>
> This is supposed to allow for thread safe access to AR objects. BUt
> I have had mixed results. Maybe you can send me a copy of your worker
> class and I can help you get it working. Also consider joining the
> backgroundrb mailing list:
>
> *Mailing list: http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>
>
> Cheers-
> -Ezra
--
Posted via http://www.ruby-forum.com/.
How do I update the backgrondrb plugin? Sorry for the noob question. Ezra Zygmuntowicz wrote:> Ben- > > Take a look in the script/backgroundrb/start file. You should see > that line in there like this: > > if CONFIG[''load_rails''] > ActiveRecord::Base.allow_concurrency = true > ActiveRecord::Base.establish_connection(YAML.load(ERB.new(IO.read("# > {RAILS_ROOT}/#{CONFIG[''database_yml'']}")).result)[CONFIG > [''environment'']]) > end > > So if you don''t see that line in the start script then you need to > add it or upgrade your version of the plugin. > > Cheers- > -Ezra-- Posted via http://www.ruby-forum.com/.
Ben- If you are not getting errors then you are good to go.Threading with ActiveRecord is a tough subject that no one seems to give a straight answer to. You can only really just test your solution with real world type loads and see if it falls on its face. But the allow_concurreny does seem to help a lot for me. To update the plugin you need to do this: delete the RAILS_ROOT/script/backgroundrb directory from your rails app. delete the RAILS_ROOT/config/backgroundrb.yml file Do not delete your workers in lib/workers. Then delete the plugin from RAILS_ROOT/vendor/plugins/backgroundrb. Then reinstall the plugin with: $ script/plugin install svn://rubyforge.org//var/svn/backgroundrb Then you just need to run the setup rake task. $ rake backgroundrb:setup Cheers- -Ezra On Jul 7, 2006, at 11:30 AM, Ben Johnson wrote:> Also, > > I have the following method in my worker class. I ran that in 6 > different threads at once. No allow_concurrency and no > mutex.synchronize > and I didn''t have a problem. They all ran over night for about 8 hours > without a problem. The updated_at field was changing every second. > Shouldn''t it lose connection to my database is 6 different threads are > accessing the database as fast as they can? > > def monitor_event(event_id) > event = Event.find event_id > @event_monitor_threads[event.id] = Thread.new(event) do |event| > while Event.exists? event.id > event.save! > end > end > end > > > Ezra Zygmuntowicz wrote: >> On Jul 6, 2006, at 3:42 PM, Ben Johnson wrote: >> >>> Do I need to do: >>> never be >>> called by backgroundrb. >>> >>> Because what if someone makes a request to >>> somedomain.com/products/create while my backgroundrb script is >>> calling >>> Product.find_all at the same time? Even though they aren''t both >>> in the >>> same process would this cause a problem? Or is synchronize ONLY >>> meant >>> for using in a single process environment where other processes are >>> irrelevant? >>> >>> Sorry if this is confusing, any help is greatly appreciated. >> >> >> Hey Ben- >> >> You have it right. You do not need to use a mutex in rails at all >> since the drb server is a separate process with its own db >> connection. So don''t worry about it on the rails side of things. I am >> still experimenting with different techniques for allowing >> concurrency with active record so you won''t need a mutex. Are you >> using the latest version of the plugin? I added this line recently: >> >> ActiveRecord::Base.allow_concurrency = true >> >> This is supposed to allow for thread safe access to AR objects. BUt >> I have had mixed results. Maybe you can send me a copy of your worker >> class and I can help you get it working. Also consider joining the >> backgroundrb mailing list: >> >> *Mailing list: http://rubyforge.org/mailman/listinfo/backgroundrb- >> devel >> >> >> Cheers- >> -Ezra > > > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > Rails mailing list > Rails@lists.rubyonrails.org > http://lists.rubyonrails.org/mailman/listinfo/rails-------------- next part -------------- An HTML attachment was scrubbed... URL: http://wrath.rubyonrails.org/pipermail/rails/attachments/20060707/797a1bf4/attachment-0001.html