I have a small app that is handling large(ish) file uploads, around 150-200Mb, it''s been running for about a year now. I''m running Rails 2.3.10, Thin 1.2.7, and Paperclip 2.3.5 Last week I went through and updated Rails from 2.3.5 to 2.3.10 and started to use the nginx upload module (http://www.grid.net.ru/nginx/ upload.en.html) to handle the uploading of the file, then pass the params to Paperclip to take some of the load off of Ruby for parsing the uploads. I thought this would solve the problem, but it hasn''t changed. It''s working well and uploads are taking place much quicker. However, after uploading a file, the memory usage of the Thin process jumps WAY up and never comes back down. Memory goes from about 46-47Mb while just surfing around the site and creating normal assets, to about 225+Mb after uploading a video and never drops. The only thing I can think of is that it may be re-parsing the file when Paperclip tries to get its metadata (content_type, size, etc.). Even if that were the case, I would expect memory to go back to normal afterwards. I''m still very new to programming, and I''m looking for suggestions on how to track down where the leak is happening. Any suggestions are greatly appreciated! Kyle Wiest -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
Sorry just wonderimg why dont you move to Rails 3.0 ? On Tue, Feb 15, 2011 at 7:05 PM, kwiest <kyle.wiest-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> I have a small app that is handling large(ish) file uploads, around > 150-200Mb, it''s been running for about a year now. > > I''m running Rails 2.3.10, Thin 1.2.7, and Paperclip 2.3.5 > > Last week I went through and updated Rails from 2.3.5 to 2.3.10 and > started to use the nginx upload module (http://www.grid.net.ru/nginx/ > upload.en.html) to handle the uploading of the file, then pass the > params to Paperclip to take some of the load off of Ruby for parsing > the uploads. I thought this would solve the problem, but it hasn''t > changed. > > It''s working well and uploads are taking place much quicker. However, > after uploading a file, the memory usage of the Thin process jumps WAY > up and never comes back down. Memory goes from about 46-47Mb while > just surfing around the site and creating normal assets, to about > 225+Mb after uploading a video and never drops. > > The only thing I can think of is that it may be re-parsing the file > when Paperclip tries to get its metadata (content_type, size, etc.). > Even if that were the case, I would expect memory to go back to normal > afterwards. > > I''m still very new to programming, and I''m looking for suggestions on > how to track down where the leak is happening. Any suggestions are > greatly appreciated! > > > Kyle Wiest > > -- > You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. > To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe@googlegroups.com. > For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en. > >-- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
I don''t know about this specific problem, but here''s some general info: This may be normal. It''s possible that the process grabbed the memory when needed and "freed" it, but held on to it for next time. If you upload a second video does it jump some more, or does it stabilize after a few uploads? Second, the memory might not get released until garbage collection happens, which could be any time. But, if every time you upload a file the memory use grows, then it''s a real leak.>> It''s working well and uploads are taking place much quicker. However, >> after uploading a file, the memory usage of the Thin process jumps WAY >> up and never comes back down. Memory goes from about 46-47Mb while >> just surfing around the site and creating normal assets, to about >> 225+Mb after uploading a video and never drops.-- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
Lorenzo, this was actually the first Rails app I ever built. I was just going in to do a bit of cleaning and didn''t have time for a full upgrade to Rails 3, but wanted to make sure I had all of the current security fixes. All of my current projects are Rails 3 and Ruby 1.9 Paul, after the first upload it keeps maintains that level. So, maybe not a leak after all? Don''t know if it makes a difference or not, but this app is running on REE 1.8.7 too. Thank you both for the feedback. Kyle On Feb 15, 8:10 pm, Paul <p...-mzPrHiy5csbYtjvyW6yDsg@public.gmane.org> wrote:> I don''t know about this specific problem, but here''s some general > info: This may be normal. It''s possible that the process grabbed the > memory when needed and "freed" it, but held on to it for next time. If > you upload a second video does it jump some more, or does it stabilize > after a few uploads? Second, the memory might not get released until > garbage collection happens, which could be any time. But, if every > time you upload a file the memory use grows, then it''s a real leak. > > > > >> It''s working well and uploads are taking place much quicker. However, > >> after uploading a file, the memory usage of the Thin process jumps WAY > >> up and never comes back down. Memory goes from about 46-47Mb while > >> just surfing around the site and creating normal assets, to about > >> 225+Mb after uploading a video and never drops.-- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.