Rails - Mar 2011 - A way to share a variable across multiple httpd threads with Rails?

Let''s say I have the following threading in my Rails web application:

    class MyController
      def my_action
        count = 0
        arr = []

        10.times do |i|
          arr[i] = Thread.new {
            sleep(rand(0)/10.0)
            Thread.current["mycount"] = count
            count += 1
          }
        end

        arr.each {|t| t.join; print t["mycount"], ", " }
        puts "count = #{count}"
      end
    end

As you can see, the ''count'' variable is shared across all
threads.

**Now,** what I want to do is **share ''count''** across
multiple httpd
requests and **allow my_action in MyController to have access to that
variable**. For instance, maybe whatever spawns the ruby process to
serve httpd process could hold the variable count in its scope, and then
the ruby processes spawned for httpd processes could then access that
variable.

Using memcached, a database, and session variables is out of the
question. Ultimately ''count'' will actually be a resource
object...an FTP
connection.

Is this possible? Perhaps using Apache/Passenger workers like at
http://stackoverflow.com/questions/4471680/a-way-to-access-common-ftp-connection-resource-pool-in-ruby-across-ajax-calls/4471758#4471758?

Example code would be appreciated.

-- 
Posted via http://www.ruby-forum.com/.

-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en.

On 9 Mar 2011, at 17:47, Chad Johnson
<lists-fsXkhYbjdPsEEoCn2XhGlw@public.gmane.org> wrote:
> Let''s say I have the following threading in my Rails web
application:
> 
>    class MyController
>      def my_action
>        count = 0
>        arr = []
> 
>        10.times do |i|
>          arr[i] = Thread.new {
>            sleep(rand(0)/10.0)
>            Thread.current["mycount"] = count
>            count += 1
>          }
>        end
This looks thread dangerous to me - i don''t think += is atomic.
> 
>        arr.each {|t| t.join; print t["mycount"], ", " }
>        puts "count = #{count}"
>      end
>    end
> 
> As you can see, the ''count'' variable is shared across all
threads.
> 
> **Now,** what I want to do is **share ''count''** across
multiple httpd
> requests and **allow my_action in MyController to have access to that
> variable**. For instance, maybe whatever spawns the ruby process to
> serve httpd process could hold the variable count in its scope, and then
> the ruby processes spawned for httpd processes could then access that
> variable.
> 
> Using memcached, a database, and session variables is out of the
> question. Ultimately ''count'' will actually be a resource
object...an FTP
> connection.
> 
> Is this possible?
I wouldn''t go down this road if I were you. For starters presumably the
objects in question will be modified by those workers, you''ll need to
synchronise access to them etc. You might also be painting yourself into a
corner for the day you need to scale beyond 1 server.

Personally, if I had to do this, I''d these objects in a separate
long-lived process (or processes) and have your controller actions talk to these
processes (exactly how depends on what you''ll be doing, you might
consider drb, message queues etc)

Fred
> Perhaps using Apache/Passenger workers like at
>
http://stackoverflow.com/questions/4471680/a-way-to-access-common-ftp-connection-resource-pool-in-ruby-across-ajax-calls/4471758#4471758?
> 
> Example code would be appreciated.
> 
> -- 
> Posted via http://www.ruby-forum.com/.
> 
> -- 
> You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
> To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe@googlegroups.com.
> For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en.
> 
-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en.

Frederick Cheung wrote in post #986539:
> On 9 Mar 2011, at 17:47, Chad Johnson
<lists-fsXkhYbjdPsEEoCn2XhGlw@public.gmane.org> wrote:
>
>>            Thread.current["mycount"] = count
>>            count += 1
>>          }
>>        end
>
> This looks thread dangerous to me - i don''t think += is atomic.
>> variable**. For instance, maybe whatever spawns the ruby process to
>> serve httpd process could hold the variable count in its scope, and
then
>> the ruby processes spawned for httpd processes could then access that
>> variable.
Just example code I pulled from someone online. I''d never actually use
this, and my stuff would be as thread-safe as possible.
> Personally, if I had to do this, I''d these objects in a separate
> long-lived process (or processes) and have your controller actions talk
> to these processes (exactly how depends on what you''ll be doing,
you
> might consider drb, message queues etc)
You''re talking about something like a daemon to act as a proxy, and
I''d communicate with it via some protocol I define? That was my thought
too, but it''s extra work, and I was trying to avoid spending the extra
time on this and just use the FTP connection directly.

So, regardless of whether this is a good or bad idea (I will assess that
later), is there any way to directly access a shared resource object?

-- 
Posted via http://www.ruby-forum.com/.

-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en.

> So, regardless of whether this is a good or bad idea (I will assess that
> later), is there any way to directly access a shared resource object?
Regardless of that: I think that passenger has a callback that is
executed before new worker instance is spawned. Maybe that could
somehow help you.

Robert Pankowecki

-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en.

On 9 Mar 2011, at 19:34, Chad Johnson
<lists-fsXkhYbjdPsEEoCn2XhGlw@public.gmane.org> wrote:
> Frederick Cheung wrote in post #986539:
>> On 9 Mar 2011, at 17:47, Chad Johnson
<lists-fsXkhYbjdPsEEoCn2XhGlw@public.gmane.org> wrote:
>> 
>>>           Thread.current["mycount"] = count
>>>           count += 1
>>>         }
>>>       end
>> 
>> This looks thread dangerous to me - i don''t think += is
atomic.
>>> variable**. For instance, maybe whatever spawns the ruby process to
>>> serve httpd process could hold the variable count in its scope, and
then
>>> the ruby processes spawned for httpd processes could then access
that
>>> variable.
> 
> Just example code I pulled from someone online. I''d never actually
use
> this, and my stuff would be as thread-safe as possible.
> 
>> Personally, if I had to do this, I''d these objects in a
separate
>> long-lived process (or processes) and have your controller actions talk
>> to these processes (exactly how depends on what you''ll be
doing, you
>> might consider drb, message queues etc)
> 
> You''re talking about something like a daemon to act as a proxy,
and
> I''d communicate with it via some protocol I define? That was my
thought
> too, but it''s extra work, and I was trying to avoid spending the
extra
> time on this and just use the FTP connection directly.
> 
I think you''d find that drb is very easy to use. 
> So, regardless of whether this is a good or bad idea (I will assess that
> later), is there any way to directly access a shared resource object?
I doubt it - while with something like passenger your workers do share initial
resources (since they are forked from the application spawner (at least by
default)), that''s a one way thing: if one worker changes the state of
the object then that happens in that process only. You do share file
descriptors, but that''s more likely to be a nuisance - if two workers
try use the same socket as the same time, chaos will ensue

Fred
> 
> -- 
> Posted via http://www.ruby-forum.com/.
> 
> -- 
> You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
> To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe@googlegroups.com.
> For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en.
> 
-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en.

On Mar 9, 2:34 pm, Chad Johnson
<li...-fsXkhYbjdPsEEoCn2XhGlw@public.gmane.org>
wrote:
> Frederick Cheung wrote in post #986539:
>
> > On 9 Mar 2011, at 17:47, Chad Johnson
<li...-fsXkhYbjdPsEEoCn2XhGlw@public.gmane.org> wrote:
>
> >>            Thread.current["mycount"] = count
> >>            count += 1
> >>          }
> >>        end
>
> > This looks thread dangerous to me - i don''t think += is
atomic.
> >> variable**. For instance, maybe whatever spawns the ruby process
to
> >> serve httpd process could hold the variable count in its scope,
and then
> >> the ruby processes spawned for httpd processes could then access
that
> >> variable.
>
> Just example code I pulled from someone online. I''d never actually
use
> this, and my stuff would be as thread-safe as possible.
>
> > Personally, if I had to do this, I''d these objects in a
separate
> > long-lived process (or processes) and have your controller actions
talk
> > to these processes (exactly how depends on what you''ll be
doing, you
> > might consider drb, message queues etc)
>
> You''re talking about something like a daemon to act as a proxy,
and
> I''d communicate with it via some protocol I define? That was my
thought
> too, but it''s extra work, and I was trying to avoid spending the
extra
> time on this and just use the FTP connection directly.
>
> So, regardless of whether this is a good or bad idea (I will assess that
> later), is there any way to directly access a shared resource object?
I don''t think there''s any *question* this is a bad idea. For
instance,
how are you planning on arbitrating the FTP connections to avoid
situations like this:

Process 1: cd foo
Process 2: cd bar
Process 1: put somefile.txt

Now somefile.txt is in the wrong place... There are similar issues
with keeping the connections alive, etc plus the *total* mess that
happens once you move beyond a single box.

In the Stackoverflow question, you mention that you''re only listing
directories. Maybe it would make more sense to write a fairly basic
proxy in a smaller framework (Sinatra maybe?) that just takes JSON
specifying what to list and spits back results? Such a server would
solve the issues mentioned above, and could even do basic caching.

--Matt Jones

-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en.