I need some help to clear up a confusion on my part. I''ve read a few things that mentioned memory leaks in Rails apps, and I''ve noticed my own Rails app memory footprint constantly increasing. Consequently the FCGI process dies eventually from hitting a process max. Now that I think about it, I think is what was causing the crashes at Dreamhost. But doesn''t Ruby take care of cleaning up memory with its garbage collector? Obviously it needs to feel that a variable is out of scope, and therefore can release the memory. So what are the causes for it not to release the objects after a Rails request has completed? And more important, how do you really hunt these down and fix these issues? Thanks for any pointers. Cheers, Brett
Brett Walker wrote:> I need some help to clear up a confusion on my part. I''ve read a few > things that mentioned memory leaks in Rails apps, and I''ve noticed my > own Rails app memory footprint constantly increasing. Consequently the > FCGI process dies eventually from hitting a process max. Now that I > think about it, I think is what was causing the crashes at Dreamhost. > > But doesn''t Ruby take care of cleaning up memory with its garbage > collector? Obviously it needs to feel that a variable is out of scope, > and therefore can release the memory. So what are the causes for it > not to release the objects after a Rails request has completed? > > And more important, how do you really hunt these down and fix these > issues? > > Thanks for any pointers. > > Cheers, > Brett > _______________________________________________ > Rails mailing list > Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org > http://lists.rubyonrails.org/mailman/listinfo/rails > >I too have had memory leak issues. I''m not using the FCGI anymore (SCGI instead). I found that making sure all objects were set to nil ( assuming your done with them) and sprinkling GC.enable (enable garbage collection) and GC.start (start garbage collection) in controller methods seems to force ruby to clean up quicker. There''s a technique to test the garbage collection of objects you create in Programming Ruby The Pragmatic Programmers’ Guide Second Edition -- John Cole
John, Do you find the SCGI is stable enough for production? I would love to run it - guess I just need to sit down with it. Are you talking about using the ObjectSpace module? I will give that a try and see what I can find. I''m hesitant to start adding GC calls to the code. Apparently on the FCGi dispatcher, you can specify the GC to be run every so many requests. But people are saying to leave that alone and use the default. Seem like adding direct GC calls is a similar situation. But maybe it will be the only thing to do. Brett On Nov 9, 2005, at 11:44 AM, John Cole wrote:> Brett Walker wrote: > >> I need some help to clear up a confusion on my part. I''ve read a >> few things that mentioned memory leaks in Rails apps, and I''ve >> noticed my own Rails app memory footprint constantly increasing. >> Consequently the FCGI process dies eventually from hitting a >> process max. Now that I think about it, I think is what was >> causing the crashes at Dreamhost. >> >> But doesn''t Ruby take care of cleaning up memory with its garbage >> collector? Obviously it needs to feel that a variable is out of >> scope, and therefore can release the memory. So what are the >> causes for it not to release the objects after a Rails request has >> completed? >> >> And more important, how do you really hunt these down and fix >> these issues? >> >> Thanks for any pointers. >> >> Cheers, >> Brett >> _______________________________________________ >> Rails mailing list >> Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org >> http://lists.rubyonrails.org/mailman/listinfo/rails >> >> > I too have had memory leak issues. I''m not using the FCGI anymore > (SCGI instead). > > I found that making sure all objects were set to nil ( assuming > your done with them) and sprinkling GC.enable (enable garbage > collection) and GC.start (start garbage collection) in controller > methods seems to force ruby to clean up quicker. > > There''s a technique to test the garbage collection of objects you > create in > > Programming Ruby > The Pragmatic Programmers’ Guide > Second Edition > > -- > John Cole > > > _______________________________________________ > Rails mailing list > Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org > http://lists.rubyonrails.org/mailman/listinfo/rails
Brett Walker wrote:> John, > > Do you find the SCGI is stable enough for production? I would love > to run it - guess I just need to sit down with it. > > Are you talking about using the ObjectSpace module? I will give that > a try and see what I can find. > > I''m hesitant to start adding GC calls to the code. Apparently on the > FCGi dispatcher, you can specify the GC to be run every so many > requests. But people are saying to leave that alone and use the > default. Seem like adding direct GC calls is a similar situation. > But maybe it will be the only thing to do. > > Brett > > On Nov 9, 2005, at 11:44 AM, John Cole wrote: > >> Brett Walker wrote: >> >>> I need some help to clear up a confusion on my part. I''ve read a >>> few things that mentioned memory leaks in Rails apps, and I''ve >>> noticed my own Rails app memory footprint constantly increasing. >>> Consequently the FCGI process dies eventually from hitting a >>> process max. Now that I think about it, I think is what was causing >>> the crashes at Dreamhost. >>> >>> But doesn''t Ruby take care of cleaning up memory with its garbage >>> collector? Obviously it needs to feel that a variable is out of >>> scope, and therefore can release the memory. So what are the causes >>> for it not to release the objects after a Rails request has completed? >>> >>> And more important, how do you really hunt these down and fix these >>> issues? >>> >>> Thanks for any pointers. >>> >>> Cheers, >>> Brett >>> _______________________________________________ >>> Rails mailing list >>> Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org >>> http://lists.rubyonrails.org/mailman/listinfo/rails >>> >>> >> I too have had memory leak issues. I''m not using the FCGI anymore >> (SCGI instead). >> >> I found that making sure all objects were set to nil ( assuming your >> done with them) and sprinkling GC.enable (enable garbage collection) >> and GC.start (start garbage collection) in controller methods seems >> to force ruby to clean up quicker. >> >> There''s a technique to test the garbage collection of objects you >> create in >> >> Programming Ruby >> The Pragmatic Programmers’ Guide >> Second Edition >> >> -- >> John Cole >> >> >> _______________________________________________ >> Rails mailing list >> Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org >> http://lists.rubyonrails.org/mailman/listinfo/rails > > > _______________________________________________ > Rails mailing list > Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org > http://lists.rubyonrails.org/mailman/listinfo/rails > >Brett, I''m still much in the proof of concept stage. I will say that I certainly find SCGI to be more stable than FCGI. Haven''t had any experience with customizing the FCGI dispatcher to cleanup. I can understand the opinion of not wanting to force garbage collection. But if the total memory consumption approaches a state where garbage collection is fired on its own (running out of memory) the cleanup time of the garbage collector will really slow things down. I just wish there was (and maybe there is) more customization in the time the garbage collection is invoked. For now when you test, I guess you watch memory consumption and try to identify memory wasteful code. I''m not aware of any tools to help along these lines. So depending on your OS I guess it''s a matter of monitoring the ruby process. Good luck, John
That''s interesting John. I''ve been doing tests with various applications (Typo, Hieraki, I2, etc.) looking for memory leaks and couldn''t replicate any. What you said below makes me think that Ruby''s GC might not be able to clear out the memory if it''s created too fast. I''ll look at some tweaks to help with this. I especially think that you shouldn''t have to sprinkle this kind of GC code all over. Zed On Wed, 09 Nov 2005 12:44:35 -0500 John Cole <cole-4EMvX6ScFxe/3pe1ocb+swC/G2K4zDHf@public.gmane.org> wrote:> I too have had memory leak issues. I''m not using the FCGI anymore > (SCGI instead). > > I found that making sure all objects were set to nil ( assuming your > done with them) and sprinkling GC.enable (enable garbage collection) > and GC.start (start garbage collection) in controller methods seems > to force ruby to clean up quicker. > > There''s a technique to test the garbage collection of objects you > create in > > Programming Ruby > The Pragmatic Programmers’ Guide > Second Edition > > -- > John Cole > > > _______________________________________________ > Rails mailing list > Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org > http://lists.rubyonrails.org/mailman/listinfo/rails
Zed A. Shaw wrote:> That''s interesting John. I''ve been doing tests with various > applications (Typo, Hieraki, I2, etc.) looking for memory leaks and > couldn''t replicate any. What you said below makes me think that Ruby''s > GC might not be able to clear out the memory if it''s created too fast. > > I''ll look at some tweaks to help with this. I especially think that > you shouldn''t have to sprinkle this kind of GC code all over. > > ZedWouldn''t you just want to run the GC after an entire request has been processed? Just run it once at the end seems like the best place for it. Blair