Hi All, I have a small ruby on rails site, using passenger and apache. The site is moderately successful. It allows people to upload photos. It seems like when people upload photos, the memory is never returned, or very late, so on a busy day I can start to use 1 GB of swap in 30 minutes. I use the attachment_fu to handle the uploads. Any ideas on how to use less memory and give the memory back quicker ? Trausti -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
Try using Enterprise Ruby (from the same people as Passenger). It can reduce your memory footprint quite a bit. However, in the case you are describing, it could just be that your app needs to be tuned. If you are doing processing on the images, you might want to put those to a background thread like delayed_job, although that process could also suck all the ram, so you might need to use monit on it to periodically restart it. However, tuning aside, if you are finding your volume is requiring more RAM, you might need to bump your hardware to support it. On Nov 29, 2009, at 5:47 PM, Trausti Thor Johannsson wrote:> Hi All, > > I have a small ruby on rails site, using passenger and apache. The > site is moderately successful. It allows people to upload photos. It > seems like when people upload photos, the memory is never returned, or > very late, so on a busy day I can start to use 1 GB of swap in 30 > minutes. > > I use the attachment_fu to handle the uploads. > > Any ideas on how to use less memory and give the memory back quicker ? > > > > Trausti > > -- > > You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. > To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe@googlegroups.com. > For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en. > >-- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
lancecarlson-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org
2009-Nov-30 08:14 UTC
Re: Massive memory eating
http://www.therailsway.com/2009/4/23/uploading-files On Nov 29, 5:47 pm, Trausti Thor Johannsson <traust...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> Hi All, > > I have a small ruby on rails site, using passenger and apache. The > site is moderately successful. It allows people to upload photos. It > seems like when people upload photos, the memory is never returned, or > very late, so on a busy day I can start to use 1 GB of swap in 30 > minutes. > > I use the attachment_fu to handle the uploads. > > Any ideas on how to use less memory and give the memory back quicker ? > > Trausti-- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
I am using enterprise ruby. Thought perhaps that was the problem. Trausti On Mon, Nov 30, 2009 at 7:13 AM, Niels Meersschaert <nmeersschaert-ee4meeAH724@public.gmane.org> wrote:> Try using Enterprise Ruby (from the same people as Passenger). It can reduce your memory footprint quite a bit. However, in the case you are describing, it could just be that your app needs to be tuned. If you are doing processing on the images, you might want to put those to a background thread like delayed_job, although that process could also suck all the ram, so you might need to use monit on it to periodically restart it. > > However, tuning aside, if you are finding your volume is requiring more RAM, you might need to bump your hardware to support it. > > > On Nov 29, 2009, at 5:47 PM, Trausti Thor Johannsson wrote: > >> Hi All, >> >> I have a small ruby on rails site, using passenger and apache. The >> site is moderately successful. It allows people to upload photos. It >> seems like when people upload photos, the memory is never returned, or >> very late, so on a busy day I can start to use 1 GB of swap in 30 >> minutes. >> >> I use the attachment_fu to handle the uploads. >> >> Any ideas on how to use less memory and give the memory back quicker ? >> >> >> >> Trausti >> >> -- >> >> You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. >> To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org >> To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org >> For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en. >> >> > > -- > > You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. > To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe@googlegroups.com. > For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en. > > >-- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
Do I just add this module in my apache, and don''t have to do anything in particular to my ruby stuff ? Trausti On Mon, Nov 30, 2009 at 9:14 AM, lancecarlson-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org <lancecarlson-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> http://www.therailsway.com/2009/4/23/uploading-files > > On Nov 29, 5:47 pm, Trausti Thor Johannsson <traust...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> > wrote: >> Hi All, >> >> I have a small ruby on rails site, using passenger and apache. The >> site is moderately successful. It allows people to upload photos. It >> seems like when people upload photos, the memory is never returned, or >> very late, so on a busy day I can start to use 1 GB of swap in 30 >> minutes. >> >> I use the attachment_fu to handle the uploads. >> >> Any ideas on how to use less memory and give the memory back quicker ? >> >> Trausti > > -- > > You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. > To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe@googlegroups.com. > For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en. > > >-- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
chris.reister-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org
2009-Nov-30 15:42 UTC
Re: Massive memory eating
One thing I could think of without seeing the code is maybe the files are being opened while being saved, but you are not closing the files? Chris On Nov 29, 2:47 pm, Trausti Thor Johannsson <traust...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> Hi All, > > I have a small ruby on rails site, using passenger and apache. The > site is moderately successful. It allows people to upload photos. It > seems like when people upload photos, the memory is never returned, or > very late, so on a busy day I can start to use 1 GB of swap in 30 > minutes. > > I use the attachment_fu to handle the uploads. > > Any ideas on how to use less memory and give the memory back quicker ? > > Trausti-- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
Good question. But I don''t do anything like open or close. I just
save the picture on upload.
Here is the add code that is when you press submit on the form
def add
if (params[:photo])
@photo = Photo.new(params[:photo])
if @photo.save
flash[:notice] = "Picture saved"
redirect_to
:controller=>''photos'',:action=>''index''
end
else
@photo = Photo.new
end
end
Trausti
On Mon, Nov 30, 2009 at 4:42 PM,
chris.reister-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org
<chris.reister-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
wrote:> One thing I could think of without seeing the code is maybe the files
> are being opened while being saved, but you are not closing the files?
>
> Chris
>
> On Nov 29, 2:47 pm, Trausti Thor Johannsson
<traust...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
> wrote:
>> Hi All,
>>
>> I have a small ruby on rails site, using passenger and apache. The
>> site is moderately successful. It allows people to upload photos. It
>> seems like when people upload photos, the memory is never returned, or
>> very late, so on a busy day I can start to use 1 GB of swap in 30
>> minutes.
>>
>> I use the attachment_fu to handle the uploads.
>>
>> Any ideas on how to use less memory and give the memory back quicker ?
>>
>> Trausti
>
> --
>
> You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
> To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe@googlegroups.com.
> For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en.
>
>
>
--
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en.
Now I just installed mod_porter and all. But when I use passenger-memory-stats I can see that my app uses 194 MB memory, I upload 1 picture and bang it uses 285 MB and 2nd picture it uses 316 MB. This is just ridiculous for an app that only does one thing, upload pictures and display thumbnails. Any ideas ? Trausti -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
On 01 Dec 2009, at 09:51, Trausti Thor Johannsson wrote:> Now I just installed mod_porter and all. But when I use passenger- > memory-stats > > I can see that my app uses 194 MB memory, I upload 1 picture and bang > it uses 285 MB and 2nd picture it uses 316 MB. > > This is just ridiculous for an app that only does one thing, upload > pictures and display thumbnails. > > Any ideas ?By the looks of it (every spawned Rails process takes about 40-ish MB), you''re wildly spawning extra Rails processes for some reason. We are using attachment_fu (and paperclip for that matter) intensively in a number of our applications and have no problems with high memory usage (except the initial RMagick hit). There must be something either in your server processes (RMagick, ImageMagick, other gem, …) that''s miscompiled or outdated or you have something around the image uploading that kills your server (threads spawning). The simple fact of the matter is that something is keeping the Passenger rails process busy and it spawns new ones to cope with new incoming requests. Best regards Peter De Berdt -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
Hi Peter, here is my gemlist actionmailer (2.3.5, 2.3.4, 2.3.3) actionpack (2.3.5, 2.3.4, 2.3.3) activerecord (2.3.5, 2.3.4, 2.3.3) activeresource (2.3.5, 2.3.4, 2.3.3) activesupport (2.3.5, 2.3.4, 2.3.3) cgi_multipart_eof_fix (2.5.0) daemons (1.0.10) fastthread (1.0.7) gem_plugin (0.2.3) mongrel (1.1.5) mysql (2.8.1) passenger (2.2.7) rack (1.0.1, 1.0.0) rails (2.3.5, 2.3.3) rake (0.8.7) rmagick (2.12.1) rubyzip (0.9.1) and these are my plugins attachment_fu (1-2 months old) easy-fckeditor 0.8.1 (not used anywhere) modporter-plugin (installed today) will_paginate 2.3.6 Thats all, I just updated passenger to 2.2.7 but just by uploading one picture, private memory grows from 10.5 MB to 81.5 (with picture size 1.5 MB) and with next picture 118 MB I never ever use find(:all). This is beyond ridiculous. There has to be something I am doing wrong. Best regards, Trausti On Tue, Dec 1, 2009 at 10:18 AM, Peter De Berdt <peter.de.berdt-LPO8gxj9N8aZIoH1IeqzKA@public.gmane.org> wrote:> > On 01 Dec 2009, at 09:51, Trausti Thor Johannsson wrote: > > Now I just installed mod_porter and all. But when I use > passenger-memory-stats > > I can see that my app uses 194 MB memory, I upload 1 picture and bang > it uses 285 MB and 2nd picture it uses 316 MB. > > This is just ridiculous for an app that only does one thing, upload > pictures and display thumbnails. > > Any ideas ? > > By the looks of it (every spawned Rails process takes about 40-ish MB), > you''re wildly spawning extra Rails processes for some reason. We are using > attachment_fu (and paperclip for that matter) intensively in a number of our > applications and have no problems with high memory usage (except the initial > RMagick hit). There must be something either in your server processes > (RMagick, ImageMagick, other gem, …) that''s miscompiled or outdated or you > have something around the image uploading that kills your server (threads > spawning). The simple fact of the matter is that something is keeping the > Passenger rails process busy and it spawns new ones to cope with new > incoming requests. > > Best regards > > Peter De Berdt > > -- > > You received this message because you are subscribed to the Google Groups > "Ruby on Rails: Talk" group. > To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > To unsubscribe from this group, send email to > rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > For more options, visit this group at > http://groups.google.com/group/rubyonrails-talk?hl=en. >-- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
On 01 Dec 2009, at 10:39, Trausti Thor Johannsson wrote:> Thats all, I just updated passenger to 2.2.7 but > > just by uploading one picture, private memory grows from 10.5 MB to > 81.5 (with picture size 1.5 MB) and with next picture 118 MB > > I never ever use find(:all). This is beyond ridiculous. There has to > be something I am doing wrong.What do your passenger-memory-stats show if I may ask (before and after uploading)? Best regards Peter De Berdt -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
Before ----- Passenger processes ----- PID VMSize Private Name ------------------------------- 14530 16.6 MB 0.5 MB /opt/ruby-enterprise-1.8.7-20090928/lib/ruby/gems/1.8/gems/passenger-2.2.7/ext/apache2/ApplicationPoolServerExecutable 0 /opt/ruby-enterprise-1.8.7-20090928/lib/ruby/gems/1.8/gems/passenger-2.2.7/bin/passenger-spawn-server /opt/ruby-enterprise-1.8.7-20090928/bin/ruby /tmp/passenger.14523 14531 100.8 MB 8.8 MB Passenger spawn server 22565 191.2 MB 7.7 MB Passenger ApplicationSpawner: /services/apache/example.com 22567 193.3 MB 7.8 MB Rails: /services/apache/example.com ### Processes: 4 ### Total private dirty RSS: 24.75 MB after 1 picture ----- Passenger processes ----- PID VMSize Private Name ------------------------------- 14530 16.6 MB 0.5 MB /opt/ruby-enterprise-1.8.7-20090928/lib/ruby/gems/1.8/gems/passenger-2.2.7/ext/apache2/ApplicationPoolServerExecutable 0 /opt/ruby-enterprise-1.8.7-20090928/lib/ruby/gems/1.8/gems/passenger-2.2.7/bin/passenger-spawn-server /opt/ruby-enterprise-1.8.7-20090928/bin/ruby /tmp/passenger.14523 14531 100.8 MB 8.8 MB Passenger spawn server 22565 191.2 MB 17.9 MB Passenger ApplicationSpawner: /services/apache/example.com 22567 286.0 MB 82.9 MB Rails: /services/apache/example.com ### Processes: 4 ### Total private dirty RSS: 110.11 MB picture 2 ----- Passenger processes ------ PID VMSize Private Name -------------------------------- 14530 16.6 MB 0.5 MB /opt/ruby-enterprise-1.8.7-20090928/lib/ruby/gems/1.8/gems/passenger-2.2.7/ext/apache2/ApplicationPoolServerExecutable 0 /opt/ruby-enterprise-1.8.7-20090928/lib/ruby/gems/1.8/gems/passenger-2.2.7/bin/passenger-spawn-server /opt/ruby-enterprise-1.8.7-20090928/bin/ruby /tmp/passenger.14523 14531 100.8 MB 8.8 MB Passenger spawn server 22565 191.2 MB 23.3 MB Passenger ApplicationSpawner: /services/apache/example.com 22567 317.6 MB 120.0 MB Rails: /services/apache/example.com ### Processes: 4 ### Total private dirty RSS: 152.59 MB Best regards, Trausti On Tue, Dec 1, 2009 at 11:31 AM, Peter De Berdt <peter.de.berdt-LPO8gxj9N8aZIoH1IeqzKA@public.gmane.org> wrote:> > On 01 Dec 2009, at 10:39, Trausti Thor Johannsson wrote: > > Thats all, I just updated passenger to 2.2.7 but > > just by uploading one picture, private memory grows from 10.5 MB to > 81.5 (with picture size 1.5 MB) and with next picture 118 MB > > I never ever use find(:all). This is beyond ridiculous. There has to > be something I am doing wrong. > > What do your passenger-memory-stats show if I may ask (before and after > uploading)? > > Best regards > > Peter De Berdt > > -- > > You received this message because you are subscribed to the Google Groups > "Ruby on Rails: Talk" group. > To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > To unsubscribe from this group, send email to > rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > For more options, visit this group at > http://groups.google.com/group/rubyonrails-talk?hl=en. >-- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
Pfew, that''s one Rails instance just eating more and more memory indeed. Have you been able to boil it down to RMagick itself. Otherwise I would suggest leaving thumbnailing out for a second and just saving the uploaded file to see if it is in fact the thumbnailing that eats your memory. We can work our way down from there. On 01 Dec 2009, at 11:56, Trausti Thor Johannsson wrote:> Before > ----- Passenger processes ----- > PID VMSize Private Name > ------------------------------- > 14530 16.6 MB 0.5 MB > /opt/ruby-enterprise-1.8.7-20090928/lib/ruby/gems/1.8/gems/ > passenger-2.2.7/ext/apache2/ApplicationPoolServerExecutable > 0 /opt/ruby-enterprise-1.8.7-20090928/lib/ruby/gems/1.8/gems/ > passenger-2.2.7/bin/passenger-spawn-server > /opt/ruby-enterprise-1.8.7-20090928/bin/ruby /tmp/passenger.14523 > 14531 100.8 MB 8.8 MB Passenger spawn server > 22565 191.2 MB 7.7 MB Passenger ApplicationSpawner: > /services/apache/example.com > 22567 193.3 MB 7.8 MB Rails: /services/apache/example.com > ### Processes: 4 > ### Total private dirty RSS: 24.75 MB > > after 1 picture > > ----- Passenger processes ----- > PID VMSize Private Name > ------------------------------- > 14530 16.6 MB 0.5 MB > /opt/ruby-enterprise-1.8.7-20090928/lib/ruby/gems/1.8/gems/ > passenger-2.2.7/ext/apache2/ApplicationPoolServerExecutable > 0 /opt/ruby-enterprise-1.8.7-20090928/lib/ruby/gems/1.8/gems/ > passenger-2.2.7/bin/passenger-spawn-server > /opt/ruby-enterprise-1.8.7-20090928/bin/ruby /tmp/passenger.14523 > 14531 100.8 MB 8.8 MB Passenger spawn server > 22565 191.2 MB 17.9 MB Passenger ApplicationSpawner: > /services/apache/example.com > 22567 286.0 MB 82.9 MB Rails: /services/apache/example.com > ### Processes: 4 > ### Total private dirty RSS: 110.11 MB > > picture 2 > > ----- Passenger processes ------ > PID VMSize Private Name > -------------------------------- > 14530 16.6 MB 0.5 MB > /opt/ruby-enterprise-1.8.7-20090928/lib/ruby/gems/1.8/gems/ > passenger-2.2.7/ext/apache2/ApplicationPoolServerExecutable > 0 /opt/ruby-enterprise-1.8.7-20090928/lib/ruby/gems/1.8/gems/ > passenger-2.2.7/bin/passenger-spawn-server > /opt/ruby-enterprise-1.8.7-20090928/bin/ruby /tmp/passenger.14523 > 14531 100.8 MB 8.8 MB Passenger spawn server > 22565 191.2 MB 23.3 MB Passenger ApplicationSpawner: > /services/apache/example.com > 22567 317.6 MB 120.0 MB Rails: /services/apache/example.com > ### Processes: 4 > ### Total private dirty RSS: 152.59 MB-- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
It consumes less memory if I don''t create thumbnails. Trausti On Tue, Dec 1, 2009 at 12:52 PM, Peter De Berdt <peter.de.berdt-LPO8gxj9N8aZIoH1IeqzKA@public.gmane.org> wrote:> Pfew, that''s one Rails instance just eating more and more memory > indeed. Have you been able to boil it down to RMagick itself. > Otherwise I would suggest leaving thumbnailing out for a second and > just saving the uploaded file to see if it is in fact the thumbnailing > that eats your memory. We can work our way down from there. > > > On 01 Dec 2009, at 11:56, Trausti Thor Johannsson wrote: > >> Before >> ----- Passenger processes ----- >> PID VMSize Private Name >> ------------------------------- >> 14530 16.6 MB 0.5 MB >> /opt/ruby-enterprise-1.8.7-20090928/lib/ruby/gems/1.8/gems/ >> passenger-2.2.7/ext/apache2/ApplicationPoolServerExecutable >> 0 /opt/ruby-enterprise-1.8.7-20090928/lib/ruby/gems/1.8/gems/ >> passenger-2.2.7/bin/passenger-spawn-server >> /opt/ruby-enterprise-1.8.7-20090928/bin/ruby /tmp/passenger.14523 >> 14531 100.8 MB 8.8 MB Passenger spawn server >> 22565 191.2 MB 7.7 MB Passenger ApplicationSpawner: >> /services/apache/example.com >> 22567 193.3 MB 7.8 MB Rails: /services/apache/example.com >> ### Processes: 4 >> ### Total private dirty RSS: 24.75 MB >> >> after 1 picture >> >> ----- Passenger processes ----- >> PID VMSize Private Name >> ------------------------------- >> 14530 16.6 MB 0.5 MB >> /opt/ruby-enterprise-1.8.7-20090928/lib/ruby/gems/1.8/gems/ >> passenger-2.2.7/ext/apache2/ApplicationPoolServerExecutable >> 0 /opt/ruby-enterprise-1.8.7-20090928/lib/ruby/gems/1.8/gems/ >> passenger-2.2.7/bin/passenger-spawn-server >> /opt/ruby-enterprise-1.8.7-20090928/bin/ruby /tmp/passenger.14523 >> 14531 100.8 MB 8.8 MB Passenger spawn server >> 22565 191.2 MB 17.9 MB Passenger ApplicationSpawner: >> /services/apache/example.com >> 22567 286.0 MB 82.9 MB Rails: /services/apache/example.com >> ### Processes: 4 >> ### Total private dirty RSS: 110.11 MB >> >> picture 2 >> >> ----- Passenger processes ------ >> PID VMSize Private Name >> -------------------------------- >> 14530 16.6 MB 0.5 MB >> /opt/ruby-enterprise-1.8.7-20090928/lib/ruby/gems/1.8/gems/ >> passenger-2.2.7/ext/apache2/ApplicationPoolServerExecutable >> 0 /opt/ruby-enterprise-1.8.7-20090928/lib/ruby/gems/1.8/gems/ >> passenger-2.2.7/bin/passenger-spawn-server >> /opt/ruby-enterprise-1.8.7-20090928/bin/ruby /tmp/passenger.14523 >> 14531 100.8 MB 8.8 MB Passenger spawn server >> 22565 191.2 MB 23.3 MB Passenger ApplicationSpawner: >> /services/apache/example.com >> 22567 317.6 MB 120.0 MB Rails: /services/apache/example.com >> ### Processes: 4 >> ### Total private dirty RSS: 152.59 MB > > -- > > You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. > To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe@googlegroups.com. > For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en. > > >-- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
On 01 Dec 2009, at 13:45, Trausti Thor Johannsson wrote:> It consumes less memory if I don''t create thumbnails.OK, let''s go one step further then and boil it down to RMagick (and not ImageMagick): install the mini_magick gem and change the thumbnail processor to minimagick explicitly and look at memory consumption. Since Minimagick uses the command line to process images, you should see no significant memory usage increase in your Rails instance. Also keep an eye on the total memory usage on your server and whether the imagemagick process stays around after the file has been processed (mogrify), it shouldn''t. Best regards Peter De Berdt -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
Does that mean I need to change any more code than is in my model ? I did set GC.start inside Photo.save in the controller. The situation does not go out of hand any more, but the app is using quite a lot of memory none the less. Trausti On Tue, Dec 1, 2009 at 2:21 PM, Peter De Berdt <peter.de.berdt-LPO8gxj9N8aZIoH1IeqzKA@public.gmane.org> wrote:> > On 01 Dec 2009, at 13:45, Trausti Thor Johannsson wrote: > > It consumes less memory if I don''t create thumbnails. > > OK, let''s go one step further then and boil it down to RMagick (and not > ImageMagick): install the mini_magick gem and change the thumbnail processor > to minimagick explicitly and look at memory consumption. Since Minimagick > uses the command line to process images, you should see no significant > memory usage increase in your Rails instance. Also keep an eye on the total > memory usage on your server and whether the imagemagick process stays around > after the file has been processed (mogrify), it shouldn''t. > > Best regards > > Peter De Berdt > > -- > > You received this message because you are subscribed to the Google Groups > "Ruby on Rails: Talk" group. > To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > To unsubscribe from this group, send email to > rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > For more options, visit this group at > http://groups.google.com/group/rubyonrails-talk?hl=en. >-- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
On 04 Dec 2009, at 11:13, Trausti Thor Johannsson wrote:> Does that mean I need to change any more code than is in my model ? > > I did set GC.start inside Photo.save in the controller. The situation > does not go out of hand any more, but the app is using quite a lot of > memory none the less.RMagick is quite memory hungry. RMagick 2 does have automatic garbage collection, but it seems you might be running into this gotcha: http://rubyforge.org/forum/forum.php?thread_id=1374&forum_id=1618 Best regards Peter De Berdt -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
On 04 Dec 2009, at 12:42, Peter De Berdt wrote:>> >> Does that mean I need to change any more code than is in my model ? >> >> I did set GC.start inside Photo.save in the controller. The >> situation >> does not go out of hand any more, but the app is using quite a lot of >> memory none the less. > > RMagick is quite memory hungry. RMagick 2 does have automatic > garbage collection, but it seems you might be running into this > gotcha: > > http://rubyforge.org/forum/forum.php?thread_id=1374&forum_id=1618 >One more note on this. Since you seem to be just thumbnailing, you could as well use another processor if memory is valuable. If you need to do extra processing like applying filters and constructing an image out of several other ones, then you''re best off with RMagick. There''s nothing more to it than making sure the processor (minimagick, imagescience, gd2, core image on macos x servers) is installed correctly and explicitly defining it in your model. Otherwise attachment_fu will see if it can use one of them in this order: @@default_processors = %w(ImageScience Rmagick MiniMagick Gd2 CoreImage). Best regards Peter De Berdt -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.