Rails - Jan 2013 - Forcing a 'process' to wait on an object specific lock

I have a synchronization problem that I am not sure how to solve in 
Ruby/Rails. I have a mental model of what I need to do and how it should 
work, but can''t find the way to do it in Ruby/Rails. (I''ve
learned all
about Mutexes and transactions and lock! but despite my research, am still 
stuck). I come from the kernel programming world, so I may be framing this 
incorrectly. 

Here''s the deal - 
1. I have a database object, call it a Meter. Meters have all sorts of 
associated objects.
2. My users can do things to meters and their associated objects via 
standard web requests.
3. I have a cron taks that runs on the server (Heroku) periodically and 
also does things to meters and their associated objects.
4. If a user is mucking about with a specific meter *or* it''s
associated
objects at the same time as the cron task is trying to muck with the same 
meter or associated objects, then there''s potential for a data
corruption
problem.

So - I need to block the cron task from mucking about with a Meter and all 
its associated objects while a user is doing so (and vice versa).

In my old kernel world, I would have defined a lock on each meter. Wherever 
I want to assure that a section of code that mucks with a meter is not 
running concurrently with another section of code that mucks with the same 
meter, I would wrap each code section that mucks with meters in a 
Lock/Unlock pair as follows:

section1:
m = get_my_meter
OSLock(m.lock)
# do section 1 stuff to meter and associated objects
OSUnlock(m.lock)

section2:
m = get_my_meter
OSLock(m.lock)
# do section 2 stuff to meter and associated objects
OSUnlock(m.lock)

The resulting behavior is that (assuming that two code sections tried to 
operate on the same meter concurrently), one would get there first and the 
second would be blocked until the first called the OSUnlock. At that time, 
the second would run and I would be assured that the code sections would 
always run in sequence for any given meter (as opposed to concurrently).

In Ruby/Rails, I see that there are mutexes, but they seem to be applicable 
to multiple threads running on the same in memory objects. In my case, the 
user request is going to load its copy of the meter from the database and 
the cron task is going to load its copy, so - mutexes don''t help. 
Transactions and pessimistic locking seemed promising, but from my 
experimentation with them they will only prevent concurrent operations on 
the specific object that is locked. Operations on associated objects, even 
if they are inside transaction blocks, can still run concurrently. So - it 
seems that the executing code is not waiting, rather, the database access, 
for the locked object only, is waiting.

I need to block the cron task from executing any further than where the 
lock is taken until the user task releases the lock (and vice versa). How 
do I do that?

Thanks,
Yoram

-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to
rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit
https://groups.google.com/d/msg/rubyonrails-talk/-/TsEJVTkVDP0J.
For more options, visit https://groups.google.com/groups/opt_out.

On Thursday, January 31, 2013 11:55:40 PM UTC,
yo...-AnuU6CR3FiHQT0dZR+AlfA@public.gmane.org wrote:
>
>
> The resulting behavior is that (assuming that two code sections tried to 
> operate on the same meter concurrently), one would get there first and the 
> second would be blocked until the first called the OSUnlock. At that time, 
> the second would run and I would be assured that the code sections would 
> always run in sequence for any given meter (as opposed to concurrently).
>
> In Ruby/Rails, I see that there are mutexes, but they seem to be 
> applicable to multiple threads running on the same in memory objects. In my
> case, the user request is going to load its copy of the meter from the 
> database and the cron task is going to load its copy, so - mutexes
don''t
> help. Transactions and pessimistic locking seemed promising, but from my 
> experimentation with them they will only prevent concurrent operations on 
> the specific object that is locked. Operations on associated objects, even 
> if they are inside transaction blocks, can still run concurrently. So - it 
> seems that the executing code is not waiting, rather, the database access, 
> for the locked object only, is waiting.
>
> I need to block the cron task from executing any further than where the 
> lock is taken until the user task releases the lock (and vice versa). How 
> do I do that?
>
You can still use a database lock on the meter as the gatekeeper to 
modifying either the meter or its associated objects (although obviously 
you need to do this in the webapp side too). In fact you don''t
necessarily
need a long lived database lock - it is sometimes easier to have a locked / 
locked_by attribute on the model, and only use an actual database lock (or 
optimistic locking) when writing to it.

Actually blocking for a long time on something is not something you really 
want to do in a web app (assuming this background processing is lengthy) - 
you''re better off failing fast and telling the use to try later

Fred
 
>
> Thanks,
> Yoram
>
-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to
rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit
https://groups.google.com/d/msg/rubyonrails-talk/-/94eZERYg6TIJ.
For more options, visit https://groups.google.com/groups/opt_out.

Thanks Fred.

You did drive right to the heart of the issue. The lock itself is indeed 
easy enough. It''s the waiting on the lock that i''m not sure
how to
implement (other than a loop that tests the dbase lock, which doesn''t
seem
right). I agree that in general, we don''t want to block threads in a
web
app. But - in this case, the code section that needs to execute without 
interruption is very brief and the likelihood of collision between the two 
processes on the same object is very small.

I may yet adopt your suggestion of just telling the user to try again 
later. It''s much more complicated if the cron task gets blocked (it
would
have to maintain a list of objects that it couldn''t process, then go
back
and process them later). But - is there a way to block a process on some 
sort of lock/semaphore, other than loop/test/loop?

On Friday, February 1, 2013 5:37:58 AM UTC-8, Frederick Cheung
wrote:
>
>
>
> On Thursday, January 31, 2013 11:55:40 PM UTC,
yo...-AnuU6CR3FiHQT0dZR+AlfA@public.gmane.org wrote:
>>
>>
>> The resulting behavior is that (assuming that two code sections tried
to
>> operate on the same meter concurrently), one would get there first and
the
>> second would be blocked until the first called the OSUnlock. At that
time,
>> the second would run and I would be assured that the code sections
would
>> always run in sequence for any given meter (as opposed to
concurrently).
>>
>> In Ruby/Rails, I see that there are mutexes, but they seem to be 
>> applicable to multiple threads running on the same in memory objects.
In my
>> case, the user request is going to load its copy of the meter from the 
>> database and the cron task is going to load its copy, so - mutexes
don''t
>> help. Transactions and pessimistic locking seemed promising, but from
my
>> experimentation with them they will only prevent concurrent operations
on
>> the specific object that is locked. Operations on associated objects,
even
>> if they are inside transaction blocks, can still run concurrently. So -
it
>> seems that the executing code is not waiting, rather, the database
access,
>> for the locked object only, is waiting.
>>
>> I need to block the cron task from executing any further than where the
>> lock is taken until the user task releases the lock (and vice versa).
How
>> do I do that?
>>
>
> You can still use a database lock on the meter as the gatekeeper to 
> modifying either the meter or its associated objects (although obviously 
> you need to do this in the webapp side too). In fact you don''t
necessarily
> need a long lived database lock - it is sometimes easier to have a locked /
> locked_by attribute on the model, and only use an actual database lock (or 
> optimistic locking) when writing to it.
>
> Actually blocking for a long time on something is not something you really 
> want to do in a web app (assuming this background processing is lengthy) - 
> you''re better off failing fast and telling the use to try later
>
> Fred
>  
>
>>
>> Thanks,
>> Yoram
>>
>
-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to
rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit
https://groups.google.com/d/msg/rubyonrails-talk/-/Mxfsg1FaIEwJ.
For more options, visit https://groups.google.com/groups/opt_out.

On Feb 1, 2013, at 8:28 AM, yoram-AnuU6CR3FiHQT0dZR+AlfA@public.gmane.org wrote:
> You did drive right to the heart of the issue. The lock itself is indeed
easy enough. It''s the waiting on the lock that i''m not sure
how to implement (other than a loop that tests the dbase lock, which
doesn''t seem right).
What you''re looking for is called a "condition variable", and
ruby has a class for that.

-- 
Scott Ribe
scott_ribe-ZCQMRMivIIdUL8GK/JU1Wg@public.gmane.org
http://www.elevated-dev.com/
(303) 722-0567 voice




-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to
rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
For more options, visit https://groups.google.com/groups/opt_out.

On Friday, February 1, 2013 3:28:57 PM UTC,
yo...-AnuU6CR3FiHQT0dZR+AlfA@public.gmane.org wrote:
>
> Thanks Fred.
>
> You did drive right to the heart of the issue. The lock itself is indeed 
> easy enough. It''s the waiting on the lock that i''m not
sure how to
> implement (other than a loop that tests the dbase lock, which
doesn''t seem
> right). I agree that in general, we don''t want to block threads in
a web
> app. But - in this case, the code section that needs to execute without 
> interruption is very brief and the likelihood of collision between the two 
> processes on the same object is very small.
>
> I may yet adopt your suggestion of just telling the user to try again 
> later. It''s much more complicated if the cron task gets blocked
(it would
> have to maintain a list of objects that it couldn''t process, then
go back
> and process them later). But - is there a way to block a process on some 
> sort of lock/semaphore, other than loop/test/loop?
>
> you should be able to block on a pessimistic lock - select ... for update 
will either acquire  the lock or timeout/deadlock. (The .lock! method does 
that for you). Don''t forget that the lock is released at the end of the
current transaction (so if there is no transaction the lock is released 
immediately.

Fred

-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to
rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit
https://groups.google.com/d/msg/rubyonrails-talk/-/pMPvNT2-9HUJ.
For more options, visit https://groups.google.com/groups/opt_out.