I was able to see a memory leak on my production server so I tried a simple test on the dev server: Fedora 8 Rails 1.2.6 MySQL 5 Gems: mysql (2.7) mongrel_cluster (1.0.5) mongrel (1.1.5) The mongrel is a cluster of 4. app/controllers/memleak_controller.rb: def mytest @users = User.find(:all) render(:text=>"DONE: "+Time.now.to_s) end The user model is very basic, no relationship. class User < ActiveRecord::Base end coontrollers/application.rb is also stripped down to bare bones. I then have a shell script that curl mytest (client/server on same host). The script is looped - so essentially a single thread request. I watch the memory usage jumped to 3.1% per mongrel_rails instance (as expected as it warms up). Then it starts to crawl up: in less then 2 hours, each mongrel_rails instance are taking up 5.6% of memory. If I stop the shell/curl script and let the system stay idle. The memory usage is never released. If I restart the script/curl script, memory usage starts climbing from where it left off. (garbage collections is not happening). Is anyone aware of any issues? Anything else I can do to make sure its not my code? (I''ve stripped down as much as I can). thanks. -- Posted via http://www.ruby-forum.com/. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
So I updated ruby from: 1.86.114-1.fc8 to 1.86.230-3.fc8 The leak got WORSE!!!! Leaking almost 3x as fast from the test above. -- Posted via http://www.ruby-forum.com/. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
On Thu, 2008-07-03 at 21:16 +0200, RR YY wrote:> So I updated ruby from: > > 1.86.114-1.fc8 > > to > > 1.86.230-3.fc8 > > The leak got WORSE!!!! Leaking almost 3x as fast from the test above.---- 1.86.230-4.fc8 should be installed if you update again If that still causes issues, then remove all the ruby packages and install the original 1.86.x that was included in Fedora-base before the ''improvements'' but note that there were some security fixes rolled into these updates. Craig --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Craig White wrote:> On Thu, 2008-07-03 at 21:16 +0200, RR YY wrote: >> So I updated ruby from: >> >> 1.86.114-1.fc8 >> >> to >> >> 1.86.230-3.fc8 >> >> The leak got WORSE!!!! Leaking almost 3x as fast from the test above. > ---- > 1.86.230-4.fc8 should be installed if you update again > > If that still causes issues, then remove all the ruby packages and > install the original 1.86.x that was included in Fedora-base before the > ''improvements'' but note that there were some security fixes rolled into > these updates. > > CraigDoes not look like 1.86.230-4.fc8 is out yet. At least not the fedora mirror i am using fedora.mirror.umoss.org Where can I pick it up? -- Posted via http://www.ruby-forum.com/. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Craig White
2008-Jul-03 21:03 UTC
Re: ****[Rails] Re: ****[Rails] Re: Fedora 8 and Mysql gem memory leak?
On Thu, 2008-07-03 at 22:43 +0200, RR YY wrote:> Craig White wrote: > > On Thu, 2008-07-03 at 21:16 +0200, RR YY wrote: > >> So I updated ruby from: > >> > >> 1.86.114-1.fc8 > >> > >> to > >> > >> 1.86.230-3.fc8 > >> > >> The leak got WORSE!!!! Leaking almost 3x as fast from the test above. > > ---- > > 1.86.230-4.fc8 should be installed if you update again > > > > If that still causes issues, then remove all the ruby packages and > > install the original 1.86.x that was included in Fedora-base before the > > ''improvements'' but note that there were some security fixes rolled into > > these updates. > > > > Craig > > Does not look like 1.86.230-4.fc8 is out yet. At least not the fedora > mirror i am using > > fedora.mirror.umoss.org > > Where can I pick it up?---- they should have hit today but apparently not...the F9 rpms did but you can always get them from koji... http://koji.fedoraproject.org/koji/buildinfo?buildID=54357 Craig --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Craig White wrote:> On Thu, 2008-07-03 at 22:43 +0200, RR YY wrote: >> >> The leak got WORSE!!!! Leaking almost 3x as fast from the test above. >> Does not look like 1.86.230-4.fc8 is out yet. At least not the fedora >> mirror i am using >> >> fedora.mirror.umoss.org >> >> Where can I pick it up? > ---- > they should have hit today but apparently not...the F9 rpms did but you > can always get them from koji... > > http://koji.fedoraproject.org/koji/buildinfo?buildID=54357 > > CraigNo good - looks like these recent ruby updates leeks memory like crazy on Fedora 8. I am going to use a diff fedora version and see if the leak is still there. (My code was originally on FC6 and ran with no leaks - so I suspect its ruby - as the recent updates seems to validate this). -- Posted via http://www.ruby-forum.com/. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Bharat Ruparel
2008-Jul-05 15:27 UTC
Re: ****[Rails] Re: ****[Rails] Re: Fedora 8 and Mysql gem m
Have you tried compiling ruby from source? I have done that and do not run into these problems. Generally, my problems were opposite in terms of IDE installations etc. However, hand compiling the source seems to have worked on both Ubuntu and Fedora (I am on 5, too lazy to upgrade). I have not benchmarked anything for leaks though. One thing that puzzles me is why do you have your Mongrels and Mongrel Clusters running different versions? -- Posted via http://www.ruby-forum.com/. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Bharat Ruparel wrote:> Have you tried compiling ruby from source? I have done that and do not > run into these problems. Generally, my problems were opposite in terms > of IDE installations etc. However, hand compiling the source seems to > have worked on both Ubuntu and Fedora (I am on 5, too lazy to upgrade). > I have not benchmarked anything for leaks though. One thing that > puzzles me is why do you have your Mongrels and Mongrel Clusters running > different versions?I don''t believe its related to hand compile vs bin release. The issue seems to be related to post FC6 (I''ve not tested FC7 - but the leak is there for FC8). I am in the process of testing on FC6 - will report on the results. As for the mongrel vs mongrel cluster versions - I don''t think their release version as in sync (someone correct me if i am wrong on this). I use gem update and it picked up the latest version of each. -- Posted via http://www.ruby-forum.com/. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
RR YY wrote:> Bharat Ruparel wrote: >> Have you tried compiling ruby from source? I have done that and do not >> run into these problems. Generally, my problems were opposite in terms >> of IDE installations etc. However, hand compiling the source seems to >> have worked on both Ubuntu and Fedora (I am on 5, too lazy to upgrade). >> I have not benchmarked anything for leaks though. One thing that >> puzzles me is why do you have your Mongrels and Mongrel Clusters running >> different versions? > > I don''t believe its related to hand compile vs bin release. > The issue seems to be related to post FC6 (I''ve not tested FC7 - but the > leak is there for FC8). I am in the process of testing on FC6 - will > report on the results. > > As for the mongrel vs mongrel cluster versions - I don''t think their > release version as in sync (someone correct me if i am wrong on this). I > use gem update and it picked up the latest version of each.Heres what I found. For Ruby 1.8.6 OR 1.8.7 (built from source): Recent Ruby changes does not play nice with Fedora 8. Memory usage is a lot higher (for the same task on FC6 - its takes up a lot more memory) AND it leaks. On Fedora 6: Memory usage is a lot smaller and no obvious memory leak is observed. The tests were done using my setup described above. -- Posted via http://www.ruby-forum.com/. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Craig White
2008-Jul-05 21:04 UTC
Re: ****[Rails] Re: ****[Rails] Re: Fedora 8 and Mysql gem m
On Sat, 2008-07-05 at 22:13 +0200, RR YY wrote:> RR YY wrote: > > Bharat Ruparel wrote: > >> Have you tried compiling ruby from source? I have done that and do not > >> run into these problems. Generally, my problems were opposite in terms > >> of IDE installations etc. However, hand compiling the source seems to > >> have worked on both Ubuntu and Fedora (I am on 5, too lazy to upgrade). > >> I have not benchmarked anything for leaks though. One thing that > >> puzzles me is why do you have your Mongrels and Mongrel Clusters running > >> different versions? > > > > I don''t believe its related to hand compile vs bin release. > > The issue seems to be related to post FC6 (I''ve not tested FC7 - but the > > leak is there for FC8). I am in the process of testing on FC6 - will > > report on the results. > > > > As for the mongrel vs mongrel cluster versions - I don''t think their > > release version as in sync (someone correct me if i am wrong on this). I > > use gem update and it picked up the latest version of each. > > Heres what I found. For Ruby 1.8.6 OR 1.8.7 (built from source): > > Recent Ruby changes does not play nice with Fedora 8. Memory usage is a > lot higher (for the same task on FC6 - its takes up a lot more memory) > AND it leaks. > > On Fedora 6: > > Memory usage is a lot smaller and no obvious memory leak is observed. > > The tests were done using my setup described above---- get the source rpm from FC-6 and rebuild it on Fedora 8 Craig --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Craig White wrote:> ---- > get the source rpm from FC-6 and rebuild it on Fedora 8 > > Craignot sure how that would help - since i build 1.8.7 from the same source for both FC6 and FC8. The leak and usage on FC8 was still there. -- Posted via http://www.ruby-forum.com/. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Bharat Ruparel
2008-Jul-06 02:48 UTC
Re: ****[Rails] Re: ****[Rails] Re: Fedora 8 and Mysql gem m
It is quite possible that it may have nothing to do with Ruby version but everything to do with the supporting libraries that are required to build ruby. They are likely to be different versions too between the releases of Fedora. Therefore what Craig is suggesting makes sense to me, i.e., take the bin files or RPMs built for FC6 which you have no problems with and deploy them in FC8. If you still experience problems then it has something to do with the FC8 environment, but if you don''t then you have squarely narrowed the problems down to libraries that were used to build ruby either by you or the suppliers of rpms/bins for FC8. Hope this helps. -- Posted via http://www.ruby-forum.com/. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Craig White
2008-Jul-06 03:01 UTC
Re: ****[Rails] Re: ****[Rails] Re: ****[Rails] Re: Fedora 8 and Mysql gem m
On Sun, 2008-07-06 at 04:48 +0200, Bharat Ruparel wrote:> It is quite possible that it may have nothing to do with Ruby version > but everything to do with the supporting libraries that are required to > build ruby. They are likely to be different versions too between the > releases of Fedora. Therefore what Craig is suggesting makes sense to > me, i.e., take the bin files or RPMs built for FC6 which you have no > problems with and deploy them in FC8. If you still experience problems > then it has something to do with the FC8 environment, but if you don''t > then you have squarely narrowed the problems down to libraries that were > used to build ruby either by you or the suppliers of rpms/bins for FC8. > Hope this helps.---- It does seem that if he is building from source on both Fedora 6 and Fedora 8 that the differences between them are the compiler and the mysql-libs. But I wonder whether he is actually re-compiling mongrel/mongrel-cluster with each change of ruby as it does seem possible that might have some impact. Craig --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Craig White wrote:> On Sun, 2008-07-06 at 04:48 +0200, Bharat Ruparel wrote: >> It is quite possible that it may have nothing to do with Ruby version >> but everything to do with the supporting libraries that are required to >> build ruby. They are likely to be different versions too between the >> releases of Fedora. Therefore what Craig is suggesting makes sense to >> me, i.e., take the bin files or RPMs built for FC6 which you have no >> problems with and deploy them in FC8. If you still experience problems >> then it has something to do with the FC8 environment, but if you don''t >> then you have squarely narrowed the problems down to libraries that were >> used to build ruby either by you or the suppliers of rpms/bins for FC8. >> Hope this helps. > ---- > It does seem that if he is building from source on both Fedora 6 and > Fedora 8 that the differences between them are the compiler and the > mysql-libs. > > But I wonder whether he is actually re-compiling mongrel/mongrel-cluster > with each change of ruby as it does seem possible that might have some > impact. > > CraigI did not recompile mongrel/cluster. Just got the latest gem. I am just going to revert back to FC6 - a lot less trouble and more stable. Thanks for the help. -- Posted via http://www.ruby-forum.com/. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
RR YY wrote:> Craig White wrote: >> On Sun, 2008-07-06 at 04:48 +0200, Bharat Ruparel wrote: >>> It is quite possible that it may have nothing to do with Ruby version >>> but everything to do with the supporting libraries that are required to >>> build ruby. They are likely to be different versions too between the >>> releases of Fedora. Therefore what Craig is suggesting makes sense to >>> me, i.e., take the bin files or RPMs built for FC6 which you have no >>> problems with and deploy them in FC8. If you still experience problems >>> then it has something to do with the FC8 environment, but if you don''t >>> then you have squarely narrowed the problems down to libraries that were >>> used to build ruby either by you or the suppliers of rpms/bins for FC8. >>> Hope this helps. >> ---- >> It does seem that if he is building from source on both Fedora 6 and >> Fedora 8 that the differences between them are the compiler and the >> mysql-libs. >> >> But I wonder whether he is actually re-compiling mongrel/mongrel-cluster >> with each change of ruby as it does seem possible that might have some >> impact. >> >> Craig > > I did not recompile mongrel/cluster. Just got the latest gem. > I am just going to revert back to FC6 - a lot less trouble and more > stable. > > Thanks for the help.Just an update. So I reverted back to 1.8.6.111-1.fc8 - this came with the default FC8 image that i am using and the leak seems to be gone (or at least not noticeable or slower). A few things I found (that may be related): String handling had changed significantly with the new ruby updates. So some of the plugins was behaving badly. Here are some areas I noticed that will not work as expected with the recent ruby updates: i) s = StringScanner.new(self.filename) unless s.exist? /\./ #need to use s.nil? instead with the new update ii) From attachment_fu. This is a biggie. "my pic.jpg" will be "my_pic.jpg" in old style BUT "_my_.jpg" in new ruby: def sanitize_filename(filename) returning filename.strip do |name| # NOTE: File.basename doesn''t work right with Windows paths on Unix # get only the filename, not the whole path name.gsub! /^.*(\\|\/)/, '''' # Finally, replace all non alphanumeric, underscore or periods with underscore name.gsub! /[^\w\.\-]/, ''_'' end end -- Posted via http://www.ruby-forum.com/. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
FYI: Confirmed there are issues with Ruby: http://www.rubyinside.com/june-2008-ruby-security-vulnerabilities-927.html -- Posted via http://www.ruby-forum.com/. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---