Rails - Aug 2006 - Making rails thread safe, priority or not?

I know there has been a lot of talk about rails and thread safety.
Maybe I''ve missed it, but I havent'' seen any of the core team
talk
about whether they plan on fixing this in the near future or not.  It
seems to me that making rails thread safe would do more to enhance
rails then any other single thing that could be done.  I''m not under
any illusions, I''m sure it would be a lot of work.  But I have to ask
myself the question.  If it''s something that will have to be done at
some point, and I think it will, doesn''t it make sense to tackle it
sooner rather than later?

So what do the users think?  Would you be willing to see work on other
things stopped to get this fixed?

Also, this isn''t a jab at the developers, I just haven''t seen
anyone
ask the question like this before.  I''ve seen lots of discussion about
how things are and particular areas in rails that are problematic, but
no statements as to what might be done about it looking towards the
future or what the average rails user thinks about it.

And I will personally volunteer some of my time to help out if the
decision is made to do something about this.

Chris

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk
-~----------~----~----~----~------~----~------~--~---

On Aug 24, 2006, at 11:14 AM, snacktime wrote:
>
> I know there has been a lot of talk about rails and thread safety.
> Maybe I''ve missed it, but I havent'' seen any of the core
team talk
> about whether they plan on fixing this in the near future or not.  It
> seems to me that making rails thread safe would do more to enhance
> rails then any other single thing that could be done.  I''m not
under
> any illusions, I''m sure it would be a lot of work.  But I have to
ask
> myself the question.  If it''s something that will have to be done
at
> some point, and I think it will, doesn''t it make sense to tackle
it
> sooner rather than later?
>
> So what do the users think?  Would you be willing to see work on other
> things stopped to get this fixed?
>
> Also, this isn''t a jab at the developers, I just haven''t
seen anyone
> ask the question like this before.  I''ve seen lots of discussion
about
> how things are and particular areas in rails that are problematic, but
> no statements as to what might be done about it looking towards the
> future or what the average rails user thinks about it.
>
> And I will personally volunteer some of my time to help out if the
> decision is made to do something about this.
>
> Chris

	Yeah I would love to see rails become thread safe as well. I have no  
idea how big a task that would be though. I think it might be  
monumental ;) I would donate some time to help with that as well. But  
I fear it would require a huge rewrite of a lot of stuf and I am not  
sure if it is worth the huge amount of work it might be. Might be  
better to just build a different thread safe ActionPack clone.


-Ezra

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk
-~----------~----~----~----~------~----~------~--~---

Here is Z''s post related to Rails, Mongrel, and thread-safety:
http://64.233.167.104/search?q=cache:Ilkzvy47LgcJ:permalink.gmane.org/gmane.comp.lang.ruby.mongrel.general/245+zed+shaw+rails+thread+safety&hl=en&gl=us&ct=clnk&cd=1

Joe

-- 
Posted via http://www.ruby-forum.com/.

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk
-~----------~----~----~----~------~----~------~--~---

On 8/24/06, Joe Ruby
<rails-mailing-list-ARtvInVfO7ksV2N9l4h3zg@public.gmane.org>
wrote:
>
> Here is Z''s post related to Rails, Mongrel, and thread-safety:
Yes I''ve seen that, good stuff:)  The challenge is that there is
probably so much to be done, that short of the core team deciding as a
group that they will make it a priority, it simply won''t get done.
Even if a heroic individual decided to step up and do all the work, it
would have to be in conjunction with new rules that didn''t let in new
code that wasn''t thread safe.

Even without being thread safe the world isn''t going to end, but if it
were thread safe, imagine how easy it would be to deploy for the
average site.

Chris

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk
-~----------~----~----~----~------~----~------~--~---

On 8/24/06, snacktime <snacktime-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
wrote:
>
> Even without being thread safe the world isn''t going to end, but
if it
> were thread safe, imagine how easy it would be to deploy for the
> average site.
Would be great for VPS/shared hosting, yes. But probably not much of
an issue for those who spend big bucks on rails..


Isak
>
> Chris
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk
-~----------~----~----~----~------~----~------~--~---

>
> > Maybe I''ve missed it, but I havent'' seen any of the
core team talk
> > about whether they plan on fixing this in the near future or not.  It
> > seems to me that making rails thread safe would do more to enhance
> > rails then any other single thing that could be done.
>
>
> Why?
>
Because then something like mongrel could just be a stand alone
solution to hosting a lot of rails apps.  No http proxy in front of it
required nor would you need mongrel clusters.  A lot of the complexity
and learning curve for new users would disappear.


>
> Thanks Chris. I''d start by removing the mutex in Mongrel and
patching up any
> guts that spill out.
>
> The greatest benefit I see is reduced memory footprint. By that standard,
> why aren''t the growing crop of shared Rails hosters pursuing it?
hosting companies don''t have an incentive.  As long as everyone is on
a level playing field and has to deal with it, what''s the incentive of
a single company to invest the time into making rails thread safe?
Memory is a lot cheaper then man hours.

Like I said it''s not the end of the world if rails isn''t
thread safe.
It''s just one of those things like ruby performance that I think is
probably going to haunt rails until it''s dealt with.  That said, I
also appreciate that the developers put their free time into creating
rails, so my opinion is just and only that, an opinion.

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk
-~----------~----~----~----~------~----~------~--~---

On 8/24/06, snacktime <snacktime-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
wrote:
>
> So what do the users think?  Would you be willing to see work on other
> things stopped to get this fixed?

We''ve resolved this under JRuby Rails deployments (mostly as
proofs-of-concept) by having multiple instances of the JRuby runtime
in-memory. It consumes a little more memory, but it allows us to scale Rails
up to as many processors as are available while also simply having extra
instances primed and ready for concurrent requests. We have not run any
scalability tests, however, and Rails support under JRuby is still a bit of
an adventure.

Ruby 2.0 is slated to have some sort of multi-VM capabilities like JRuby,
where multiple independent instances of Ruby can co-exist within the same
process. Also, whytheluckystiff has been working on sandboxing code that
allows a similar feature under Ruby 1.8. However both cases are still in
progress, and the JRuby option seems to be the most functional right now.

It''s also worth noting that other than some minimal memory savings,
multi-threaded Rails wouldn''t really gain you much under Ruby 1.8 since
threads are green in 1.8 (i.e. all Ruby threads are pumped by only one CPU
thread). Using stock Ruby you really need to have multiple processes to
scale properly.

-- 
Contribute to RubySpec! @ www.headius.com/rubyspec
Charles Oliver Nutter @ headius.blogspot.com
Ruby User @ ruby.mn
JRuby Developer @ www.jruby.org
Application Architect @ www.ventera.com


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk
-~----------~----~----~----~------~----~------~--~---

On 8/24/06, snacktime <snacktime-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
wrote:
>
>
> > > Maybe I''ve missed it, but I havent'' seen any of
the core team talk
> > > about whether they plan on fixing this in the near future or not.
It
> > > seems to me that making rails thread safe would do more to
enhance
> > > rails then any other single thing that could be done.
> >
> > Why?
> >
>
> Because then something like mongrel could just be a stand alone
> solution to hosting a lot of rails apps.  No http proxy in front of it
> required nor would you need mongrel clusters.  A lot of the complexity
> and learning curve for new users would disappear.

Indeed - it''d also allow dev and production on the same platform. But
why is
this more important than any other single thing that could be done?

Note that thread safety doesn''t help much when the database
doesn''t have a
nonblocking API or the Ruby driver doesn''t support it.

>
> > Thanks Chris. I''d start by removing the mutex in Mongrel and
patching up
> any
> > guts that spill out.
> >
> > The greatest benefit I see is reduced memory footprint. By that
> standard,
> > why aren''t the growing crop of shared Rails hosters pursuing
it?
>
> hosting companies don''t have an incentive.  As long as everyone is
on
> a level playing field and has to deal with it, what''s the
incentive of
> a single company to invest the time into making rails thread safe?
> Memory is a lot cheaper then man hours.

This is true: your app gets a hit and has to be swapped in to serve it.
Shared hosting is insane.


Like I said it''s not the end of the world if rails isn''t
thread safe.
> It''s just one of those things like ruby performance that I think
is
> probably going to haunt rails until it''s dealt with.  That said, I
> also appreciate that the developers put their free time into creating
> rails, so my opinion is just and only that, an opinion.

The cost/benefit just isn''t there yet, though a motivated hacker may
find
sufficient reward in improved code quality alone. Otherwise, it''s on
the
calendar pending Ruby2 + native threads.

As long as we''re slicing pie from the sky, how about _why-sandboxing in
development mode, or having YARV spawn a VM per production app? Get mod_ruby
back on the playing field..

Best,
jeremy


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk
-~----------~----~----~----~------~----~------~--~---

> It''s also worth noting that other than some minimal memory
savings,
> multi-threaded Rails wouldn''t really gain you much under Ruby 1.8
since
> threads are green in 1.8 (i.e. all Ruby threads are pumped by only one CPU
> thread). Using stock Ruby you really need to have multiple processes to
> scale properly.
But that''s only a small percentage of rails sites that even need duel
cpu''s.  We have mod perl apps running on single cpu boxes that
don''t
even breathe hard at 400,000 hits per day, and those servers are 2
years old.  Fast drives and lots of memory though.  Plus, even if
things were thread safe you would still probably be running one
process per application, so for hosting companies they would still be
able to make use of more than one cpu.

One thing that definitely hasn''t been tested is how much you could
really gain by being able to run rails multithreaded, or where
diminishing returns would start to kick in.  For all I know that could
happen at 3 threads or 20.

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk
-~----------~----~----~----~------~----~------~--~---

Jeremy Kemper wrote:
> (posting to -core as well.)
> 
> On 8/24/06, snacktime <snacktime@gmail.com> wrote:
>> I know there has been a lot of talk about rails and thread safety.
> 
> 
> Could you point me to these discussions?
> 
This, from Zed, is the most specific information I''m aware of:

   http://rubyforge.org/pipermail/mongrel-users/2006-May/000302.html

:-(

regards

   Justin

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Core" group.
To post to this group, send email to rubyonrails-core@googlegroups.com
To unsubscribe from this group, send email to
rubyonrails-core-unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/rubyonrails-core
-~----------~----~----~----~------~----~------~--~---

Ok. I guess the real basic stuff would be to find out:
  * how can you break thread safety.
And then:
  * audit the code trying to stop thread dangerous behaviour. Raising 
stuff when someone does something wrong.

I know this sound too "academic" and little practical, but, to me, 
proving by exausting the currently known problems won''t get us to
htread
safety.


Just 2 cents. Ignore them if you disagree.

Cheers!

-- 
Posted via http://www.ruby-forum.com/.

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk
-~----------~----~----~----~------~----~------~--~---

I think that making rails thread-safe would greatly expand its
usability.

Today, having background daemon threads is a pain in the azz because you
need to start and control other ruby processes.

There are even some things completely impossible to do with rails, like
as event-driven AJAX, or AJAX observer (like that GMail chat)... Which,
personally, I think is one of the big deals of ajax to deliver
instantaneous responses...

-- 
Posted via http://www.ruby-forum.com/.

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---

Rodrigo Rosauro wrote:
> I think that making rails thread-safe would greatly expand its
> usability.
A ''thread'' is like a ''goto''. It''s
useful in a pinch, but it''s really bad
structure.
> Today, having background daemon threads is a pain in the azz because you
> need to start and control other ruby processes.
If you brought them closer to the Rails that serves pages, they would donate 
to it their fragility. They work best in other processes, with DRb between 
them and you, for insulation.

If you need to query them too often, then you probably have a latent 
time-sliced architecture. Converting that into something truly event-driven 
would probably improve its design, and make it leaner and more robust.

Next, threads and processes are a pain in the nuts to unit test. The blame 
doesn''t belong to the tests. When you thread, and when you use
semaphores to
synchronize your threads, you commit the ultimate failure of encapsulation. 
You make one thread sensitive to variations in the tiniest private details 
of another thread''s timing.
> There are even some things completely impossible to do with rails, like
> as event-driven AJAX, or AJAX observer (like that GMail chat)... Which,
> personally, I think is one of the big deals of ajax to deliver
> instantaneous responses...
I currently use BackgrounDRb in two projects. For both of them, I first 
tried the non-detached version. They both didn''t work. One, zipping up
a
bunch of files, took too long. So I let the user click, start a task, start 
a spinning graphic on the user interface, and wait for the task to end. A 
periodically_call_remote polls the task. When it finishes, the remote call 
returns JavaScript that sets a small IFRAME''s src attribute to the
server
location of the zipped file. That pops up the Save File dialog 
automatically.

For the other project, a game, I need abstract timing events to update 
different web browsers simultaneously, in real-time. Again, I first tried 
that without a background process, and it was too slow and clumsy. So I put 
the event timer, and its state table, into a BackgrounDRb process, and I use 
Juggernaut to send new JavaScript updates to both browsers.

GMail is similar. They hacked their server to leave a TCP channel open, 
providing "server push". Don''t do that if you are not Google,
and don''t have
a horde of engineers to hack your server. Juggernaut is a nice compromise if 
you already use Flash. It uses Flash to leave that wire open, enabling 
server-push.

And never think for a second that a combination of BackgrounDRb and 
traditional Ajax will be more responsive than old-school Ajax. Speaking from 
personal experience, you will get complaints from your ISP that you are 
using too much CPU time on their server!

-- 
  Phlip
  http://flea.sourceforge.net/PiglegToo_1.html 


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---