My application is going to have a work queue of scheduled tasks stored in the DB. This work queue will be shared across multiple Rails app servers. I need a way to ensure that no two servers get the same jobs if multiple servers pop jobs off the top of the queue simultaneously. Is there a fairly simple way to acheive this? Or do I need to come up with some fancy secondary server to dispatch and marshall (serialize) job assignments? Thanks! - Don -- Posted via http://www.ruby-forum.com/.
On May 9, 2006, at 9:49 AM, Don Stocks wrote:> My application is going to have a work queue of scheduled tasks stored > in the DB. This work queue will be shared across multiple Rails app > servers. I need a way to ensure that no two servers get the same jobs > if multiple servers pop jobs off the top of the queue simultaneously. > > Is there a fairly simple way to acheive this? Or do I need to come up > with some fancy secondary server to dispatch and marshall (serialize) > job assignments?Attempt to lock the job after popping it from the queue: class Job < AR::Base #self.locking_column = ''lock_version'' belongs_to :queue end class Queue < AR::Base has_many :pending_jobs, :class_name => Job, :conditions => ''pending=true'' def pop if target = pending_jobs.find(:first) target.pending = false target.save! target end rescue ActiveRecord::StaleObjectError # Grab another if someone else locked ours first. retry end end jeremy
Fantastic input Jeremy. Thanks! That really solves a lot of problems. I''ll have to experiment and give it a test drive. If it works well in a very busy environment then I''ll be set. Thanks a ton! By the way, there may be a lot of jobs and many servers accessing the queue. What are your thoughts on the most efficient way to fetch multiple jobs at a time? - Don -- Posted via http://www.ruby-forum.com/.
I''m not sure about your application but maybe you should look into Distributed Ruby and/or Rinda. It provides a lot of functionality for working with queues and tasks in a distributed space. Matt -- Posted via http://www.ruby-forum.com/.
Matt Bauer wrote:> I''m not sure about your application but maybe you should look into > Distributed Ruby and/or Rinda. It provides a lot of functionality for > working with queues and tasks in a distributed space. > > MattI looked into and experimented with Rinda and DRb but I couldn''t figure out how it would solve my original problem without introducing a single point of failure. If all my Rails servers used the same intance of my Queue via DRb, then I would need to engineer a contignecy for failover. And that would probably mean multiple DRb servers, in which case I''d need to do what Jeremy suggests (or something like it) anyway. Additionaly, since my queue is comprised of fairly dynamic ActiveRecord objects, they go out of sync with the locally cached objects published via DRb. So the DRb server would need to continually refresh the objects in its cache from the DB. Although Jeremy''s solution is less sexy than a Distributed Ruby design, it seems like it''s less complex. I''m brand new to Ruby / Rails / DRb / Rinda. So I''m probably missing a lot here. Any insight is always appreciated! Thanks, Don -- Posted via http://www.ruby-forum.com/.
On May 9, 2006, at 12:09 PM, Don Stocks wrote:> By the way, there may be a lot of jobs and many servers accessing the > queue. What are your thoughts on the most efficient way to fetch > multiple jobs at a time?I''d add a :lock option to AR::Base#find so you could: class Job < AR::Base belongs_to :owner def self.claim!(owner, n = 10) # select * from jobs where owner_id is null limit 10 for update jobs = find(:all, :conditions => ''owner_id is null'', :limit => n, :lock => ''for update'') # update jobs set owner_id=1 where id in (1,2,3,4,5) update_all([''owner_id=?'', owner], [''id in (?)'', jobs.map { |j| j.id }]) jobs end end select ... for update gives you an exclusive lock on the selected rows so others can''t set themselves as owner before you do. Best, jeremy