I''m involved in upgrading a large Rails app from 2.3.4 to 3.0.7. We noticed the Rails 3 version of the app uses much more memory (causing monit to continually restart our mongrels). Using REE 1.8.7. After many experiments and lots of acrobatics with memprof (described in detail at http://railski.blogspot.com/2011/05/rails3-leaks-more-memory.html with code and heap dumps at https://github.com/pkmiec/leakage), I believe the problem is related to lazy initialization. Rails 3 defers lots of initialization until later in order to speed up start up times. While handling requests, Rails initializes parts of the app needed to fulfill the request. For example, a model is required during start up (assuming it''s in the eager load path), but its columns are not known and attribute methods not created until they are needed (for example when an instance of the model is created). What I observed (and confirmed with memprof) is that controller instances (and everything related to them) stick around on the heap if some lazy initialization occurred during the request. So far I''ve identified attribute methods (in particular define_attribute_methods), controller''s view_context_class, and controller''s config to cause leaks. What these things have in common is scopes that are assigned to global or class level variables and eigen / singleton class madness. I can force the initialization to occur in an after initialize hook (call define_attribute_methods and protected_attributes on every model, call view_context_class and config on every controller) and prevent any controller instances from hanging around the heap. This tricks works in the simplest Rails 3 app as well as in the large app that''s going from 2.3.4 to 3.0.7. Looking at the heap data, I can''t find a relationship between the "things" involved in lazy initialization and "things" related to the controller instance other than they share the same thread that created them. This leads me to believe the problem I''m seeing is really a ruby vm problem. I''ve tried different version of ruby 1.8.7 and they all exhibit the same behavior. Am I missing something obvious? Can someone explain why pre-loading causes no controller instances to be left hanging on the heap (and therefore no memory issues with the large app)? I''d like to repeat my experiments with Ruby 1.9, but memprof doesn''t currently support 1.9. Is there another approach to inspect the heap with 1.9? -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To post to this group, send email to rubyonrails-core@googlegroups.com. To unsubscribe from this group, send email to rubyonrails-core+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/rubyonrails-core?hl=en.
Hello :) I have same issue on two applications.
Each request eat 50-70 mb of memory. (on development and production
modes)
I use memprof-tracer and find some problem in /activesupport-3.0.7/lib/
active_support/dependencies.rb line 484:
trace = caller.reject {|l| l =~ %r{#{Regexp.escape(__FILE__)}}}
when i comment it like this: (i know this is stupid):
trace = caller#.reject {|l| l =~ %r{#{Regexp.escape(__FILE__)}}}
and my production mode start to work well! But not a development.
Today i will continue my exploration :))
--
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Core" group.
To post to this group, send email to rubyonrails-core@googlegroups.com.
To unsubscribe from this group, send email to
rubyonrails-core+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/rubyonrails-core?hl=en.
I rolled up my sleeves and gave gdb a go. I believe I found the link between the hanging controller instances and Rails'' lazy initialization. Ultimately, it does smell like a ruby bug, but perhaps someone with more knowledge can comment on it. You can see my follow up http://railski.blogspot.com/2011/05/proving-rails-memory-leak-with-gdb.html. On May 17, 10:14 pm, Paul <pkm...@gmail.com> wrote:> I''m involved in upgrading a large Rails app from 2.3.4 to 3.0.7. We > noticed the Rails 3 version of the app uses much more memory (causing > monit to continually restart our mongrels). Using REE 1.8.7. > > After many experiments and lots of acrobatics with memprof (described > in detail athttp://railski.blogspot.com/2011/05/rails3-leaks-more-memory.html > with code and heap dumps athttps://github.com/pkmiec/leakage), I > believe the problem is related to lazy initialization. > > Rails 3 defers lots of initialization until later in order to speed up > start up times. While handling requests, Rails initializes parts of > the app needed to fulfill the request. For example, a model is > required during start up (assuming it''s in the eager load path), but > its columns are not known and attribute methods not created until they > are needed (for example when an instance of the model is created). > > What I observed (and confirmed with memprof) is that controller > instances (and everything related to them) stick around on the heap if > some lazy initialization occurred during the request. So far I''ve > identified attribute methods (in particular define_attribute_methods), > controller''s view_context_class, and controller''s config to cause > leaks. What these things have in common is scopes that are assigned to > global or class level variables and eigen / singleton class madness. > > I can force the initialization to occur in an after initialize hook > (call define_attribute_methods and protected_attributes on every > model, call view_context_class and config on every controller) and > prevent any controller instances from hanging around the heap. This > tricks works in the simplest Rails 3 app as well as in the large app > that''s going from 2.3.4 to 3.0.7. > > Looking at the heap data, I can''t find a relationship between the > "things" involved in lazy initialization and "things" related to the > controller instance other than they share the same thread that created > them. This leads me to believe the problem I''m seeing is really a ruby > vm problem. I''ve tried different version of ruby 1.8.7 and they all > exhibit the same behavior. > > Am I missing something obvious? Can someone explain why pre-loading > causes no controller instances to be left hanging on the heap (and > therefore no memory issues with the large app)? > > I''d like to repeat my experiments with Ruby 1.9, but memprof doesn''t > currently support 1.9. Is there another approach to inspect the heap > with 1.9?-- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To post to this group, send email to rubyonrails-core@googlegroups.com. To unsubscribe from this group, send email to rubyonrails-core+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/rubyonrails-core?hl=en.
maybe some members of core team can make attention on this problem? -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To post to this group, send email to rubyonrails-core@googlegroups.com. To unsubscribe from this group, send email to rubyonrails-core+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/rubyonrails-core?hl=en.