Hi, A quick search didn''t explain what fastthread is, and how it relates to mongrel. Why would I want to install fastthread? Thanks, Joe
On 1/4/07, Joe Van Dyk <joevandyk at gmail.com> wrote:> Hi, > > A quick search didn''t explain what fastthread is, and how it relates > to mongrel. Why would I want to install fastthread? > > Thanks, > Joehttp://moonbase.rydia.net/mental/blog/programming/fastthread.html (3rd google result for ''ruby fastthread'') -- Rick Olson http://weblog.techno-weenie.net http://mephistoblog.com
On Thu, 4 Jan 2007 00:55:41 -0800 "Joe Van Dyk" <joevandyk at gmail.com> wrote:> Hi, > > A quick search didn''t explain what fastthread is, and how it relates > to mongrel. Why would I want to install fastthread?Hi Joe, long answer is that ruby has a bug in how the array works and how threads are garbage collected. When the current Ruby thread locking primitives operate on threads, they tend to store idle ones in arrays. As the existing thread locking mechanisms pull the threads off the array they never get garbage collected and that causes a nasty leak. At the time I was testing this, me and Bradley Taylor narrowed the problem down and produced these two scripts: http://pastie.caboo.se/10194 http://pastie.caboo.se/10317 The first one shows the leak by using Mutex (with graph), the second showed proper GC operation by using Sync (with graph). Still wasn''t enough because several high profile idiots on ruby-lang tried to claim the test was invalid but couldn''t offer a reason why the first script leaked and the second didn''t. Later, after much arguing, calling me an idiot, debating the merits of OS memory allocation, and other stupidities, we find out that Eric Mahurin fixed the whole problem a year ago and it was missed: http://blade.nagaokgaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/5861 Thankfully, there''s smart cool people like mentalguy who can make improvements and provide them as gems. What mentalguy did is re-create all of the Ruby thread locking gear in a C extension called fastthread. With fastthread we can provide a modest speed boost as well as fix the memory leaks Ruby has by just including a gem. He''s pushing for it to become the new thread locking api in Ruby, so hopefully he can get it in without much fuss. I''m telling people that you have to use fastthread, but it''s still external so if you run into problems or don''t see an improvements then you don''t have to run it. Mongrel has fastthread as a gem dependency, but it will run it only if it''s installed. So, you can install it, try it out, and then remove the gem if it''s a problem. And that''s the whole story. Enjoy! -- Zed A. Shaw, MUDCRAP-CE Master Black Belt Sifu http://www.zedshaw.com/ http://www.awprofessional.com/title/0321483502 -- The Mongrel Book http://mongrel.rubyforge.org/ http://www.lingr.com/room/3yXhqKbfPy8 -- Come get help.
On Thu, 2007-01-04 at 07:05 -0800, Zed A. Shaw wrote:> What mentalguy did is re-create all of the Ruby thread locking gear in > a C extension called fastthread. With fastthread we can provide a > modest speed boost as well as fix the memory leaks Ruby has by just > including a gem. He''s pushing for it to become the new thread locking > api in Ruby, so hopefully he can get it in without much fuss.Interestingly, the original reason for fastthread was to help wean people off of Thread.critical/Thread.exclusive by making Mutex#lock/Mutex#synchronized competitive with it performance-wise. Fixing the memory leaks was a rather fortunate side-effect. :) -mental -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://rubyforge.org/pipermail/mongrel-users/attachments/20070104/f04caf75/attachment.bin
On Thu, 2007-01-04 at 07:05 -0800, Zed A. Shaw wrote:> Later, after much arguing, calling me an idiot, debating the merits of > OS memory allocation, and other stupidities, we find out that Eric > Mahurin fixed the whole problem a year ago and it was missed: > > http://blade.nagaokgaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/5861Has his patch been followed up on now, btw? I''d hate to see it continue to go to waste. -mental -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://rubyforge.org/pipermail/mongrel-users/attachments/20070104/07679e89/attachment.bin
On 1/4/07, Zed A. Shaw <zedshaw at zedshaw.com> wrote:> On Thu, 4 Jan 2007 00:55:41 -0800 > "Joe Van Dyk" <joevandyk at gmail.com> wrote: > > > Hi, > > > > A quick search didn''t explain what fastthread is, and how it relates > > to mongrel. Why would I want to install fastthread? > > Hi Joe, long answer is that ruby has a bug in how the array works and how threads are garbage collected. When the current Ruby thread locking primitives operate on threads, they tend to store idle ones in arrays. As the existing thread locking mechanisms pull the threads off the array they never get garbage collected and that causes a nasty leak. > > At the time I was testing this, me and Bradley Taylor narrowed the problem down and produced these two scripts: > > http://pastie.caboo.se/10194 > http://pastie.caboo.se/10317 > > The first one shows the leak by using Mutex (with graph), the second showed proper GC operation by using Sync (with graph). Still wasn''t enough because several high profile idiots on ruby-lang tried to claim the test was invalid but couldn''t offer a reason why the first script leaked and the second didn''t. > > Later, after much arguing, calling me an idiot, debating the merits of OS memory allocation, and other stupidities, we find out that Eric Mahurin fixed the whole problem a year ago and it was missed: > > http://blade.nagaokgaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/5861 > > Thankfully, there''s smart cool people like mentalguy who can make improvements and provide them as gems. What mentalguy did is re-create all of the Ruby thread locking gear in a C extension called fastthread. With fastthread we can provide a modest speed boost as well as fix the memory leaks Ruby has by just including a gem. He''s pushing for it to become the new thread locking api in Ruby, so hopefully he can get it in without much fuss. > > I''m telling people that you have to use fastthread, but it''s still external so if you run into problems or don''t see an improvements then you don''t have to run it. Mongrel has fastthread as a gem dependency, but it will run it only if it''s installed. So, you can install it, try it out, and then remove the gem if it''s a problem. > > And that''s the whole story. Enjoy!Thanks for the explanation! Joe