We''re running rails + lighttpd at zvents.com. We''re starting 10 dispatch.fcgi processes when lighttpd is launched. At first, all 10 processes have approximately the same memory utilization (approx 33 MB each). After a day or so, I notice that 1 or 2 processes have high memory utilization (approx 100 MB) but the rest are still at 33MB. See the output of top (ordered by memory utilization) below. This makes me wonder if only 2 of the 10 dispatch.fcgis are actually in use. Is anyone else seeing this? Is this a cause for concern? Thanks in advance for your help, Tyler PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 23967 root 16 0 206m 103m 4444 S 0.0 5.2 0:37.58 dispatch.fcgi 23966 root 16 0 205m 103m 4460 S 0.0 5.1 0:43.58 dispatch.fcgi 23968 root 16 0 80980 37m 2940 S 0.0 1.9 0:03.61 dispatch.fcgi 23958 root 25 0 71656 33m 2584 S 0.0 1.6 0:01.13 dispatch.fcgi 23959 root 25 0 71656 33m 2584 S 0.0 1.6 0:01.10 dispatch.fcgi 23960 root 25 0 71656 33m 2584 S 0.0 1.6 0:01.11 dispatch.fcgi 23961 root 25 0 71656 33m 2584 S 0.0 1.6 0:01.10 dispatch.fcgi 23962 root 25 0 71656 33m 2584 S 0.0 1.6 0:01.10 dispatch.fcgi 23963 root 25 0 71656 33m 2584 S 0.0 1.6 0:01.10 dispatch.fcgi 23964 root 25 0 71656 33m 2584 S 0.0 1.6 0:01.10 dispatch.fcgi 23965 root 25 0 71656 33m 2584 S 0.0 1.6 0:01.09 dispatch.fcgi
On 10/16/05, Tyler Kovacs <tyler.kovacs-PG3JTQmzGg7QT0dZR+AlfA@public.gmane.org> wrote:> > We''re running rails + lighttpd at zvents.com <http://zvents.com>. We''re > starting 10 > dispatch.fcgi processes when lighttpd is launched. At first, all 10 > processes have approximately the same memory utilization (approx 33 MB > each). After a day or so, I notice that 1 or 2 processes have high > memory utilization (approx 100 MB) but the rest are still at 33MB. > See the output of top (ordered by memory utilization) below. This > makes me wonder if only 2 of the 10 dispatch.fcgis are actually in > use. Is anyone else seeing this? Is this a cause for concern? >Why do the top three processes have a higher priority? That''s what making them soak up all the requests. Chris _______________________________________________ Rails mailing list Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org http://lists.rubyonrails.org/mailman/listinfo/rails
Also, any reason why they are running as root? That''s begging for a disaster. Chris _______________________________________________ Rails mailing list Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org http://lists.rubyonrails.org/mailman/listinfo/rails
> On 10/16/05, Tyler Kovacs<tyler.kovacs-PG3JTQmzGg7QT0dZR+AlfA@public.gmane.org> wrote:> We''re running rails + lighttpd at zvents.com. We''re starting 10dispatch.fcgiprocesses when lighttpd is launched. At first, all 10processes have approximately the same memory utilization (approx 33 MB> each). After a day or so, I notice that 1 or 2 processes have highmemoryutilization (approx 100 MB) but the rest are still at 33MB.See the output of top (ordered by memory utilization) below. Thismakes me wonder if only 2 of the 10> dispatch.fcgis are actually inuse. Is anyone else seeing this? Is this acause for concern?> > Why do the top three processes have a higher priority? That''s what makingthem soak up all the requests. Bizarre. I hadn''t even noticed that before. We''re definitely not doing anything explicit to set the priority. It appears that the linux scheduler is doing this on its own: "Linux builds on this idea and provides dynamic priority-based scheduling. This concept begins with the initial base priority, and then enables the scheduler to increase or decrease the priority dynamically to fulfill scheduling objectives. For example, a process that is spending more time waiting on I/O than running is clearly I/O bound. Under Linux, it receives an elevated dynamic priority. As a counterexample, a process that continually uses up its entire timeslice is processor bound—it would receive a lowered dynamic priority." Guess I''ll look into how to disable this - or maybe we''ll just renice them through cron.> Also, any reason why they are running as root? That''s begging for a disaster.Err... legacy reasons. Time to change that.
On Oct 16, 2005, at 7:07 PM, Tyler Kovacs wrote:>> Also, any reason why they are running as root? That''s begging for >> a disaster. >> > > Err... legacy reasons. Time to change that.Say that twice! -- -- Tom Mornini
Julian ''Julik'' Tarkhanov
2005-Oct-17 03:01 UTC
Re: uneven dispatch.fcgi memory utilization
On 17-okt-2005, at 1:23, Tyler Kovacs wrote:> We''re running rails + lighttpd at zvents.com. We''re starting 10 > dispatch.fcgi processes when lighttpd is launched. At first, all 10 > processes have approximately the same memory utilization (approx 33 MB > each). After a day or so, I notice that 1 or 2 processes have high > memory utilization (approx 100 MB) but the rest are still at 33MB. > See the output of top (ordered by memory utilization) below. This > makes me wonder if only 2 of the 10 dispatch.fcgis are actually in > use. Is anyone else seeing this? Is this a cause for concern?I think it is quite possible that only three of them ever get used (which would mean that you actually don''t need the others). They are swapped by lighttpd - when one is busy, the other gets used etc. and I guess the first and the second one are both free when the third one is used, so they just get used instead of the rest. In general 10 FCGIs is for, like, er. a very very big site. As for the ones that have high memory utilization, check if you are running them in ''development''. It is notoriously leaky on FCGI. If even that is OK, check your actions and the parameter in your dispatch.fcgi (it doesn''t force garbage collection by default). -- Julian "Julik" Tarkhanov
On Oct 16, 2005, at 9:01 PM, Julian ''Julik'' Tarkhanov wrote:> On 17-okt-2005, at 1:23, Tyler Kovacs wrote: > >> We''re running rails + lighttpd at zvents.com. We''re starting 10 >> dispatch.fcgi processes when lighttpd is launched. At first, all 10 >> processes have approximately the same memory utilization (approx >> 33 MB >> each). After a day or so, I notice that 1 or 2 processes have high >> memory utilization (approx 100 MB) but the rest are still at 33MB. >> See the output of top (ordered by memory utilization) below. This >> makes me wonder if only 2 of the 10 dispatch.fcgis are actually in >> use. Is anyone else seeing this? Is this a cause for concern? > > I think it is quite possible that only three of them ever get used > (which would mean that you actually > don''t need the others). They are swapped by lighttpd - when one is > busy, the other gets used etc. and I guess > the first and the second one are both free when the third one is > used, so they just get used instead of the rest. > In general 10 FCGIs is for, like, er. a very very big site. > > As for the ones that have high memory utilization, check if you are > running them in ''development''. It is notoriously leaky on FCGI. If > even that is OK, check your actions and the parameter in your > dispatch.fcgi (it doesn''t force garbage collection by default).Just to clarify (and expand upon) this last note: FCGI in development mode leaks Proc objects in 0.13.1. This _only_ happens in development mode. Also, this leak has been fixed in the latest Rails and should no longer be an issue as of 0.14. - Jamis
.14?! Not 1.0? ;) Jamis Buck wrote:> On Oct 16, 2005, at 9:01 PM, Julian ''Julik'' Tarkhanov wrote: > >> On 17-okt-2005, at 1:23, Tyler Kovacs wrote: >> >>> We''re running rails + lighttpd at zvents.com. We''re starting 10 >>> dispatch.fcgi processes when lighttpd is launched. At first, all 10 >>> processes have approximately the same memory utilization (approx 33 MB >>> each). After a day or so, I notice that 1 or 2 processes have high >>> memory utilization (approx 100 MB) but the rest are still at 33MB. >>> See the output of top (ordered by memory utilization) below. This >>> makes me wonder if only 2 of the 10 dispatch.fcgis are actually in >>> use. Is anyone else seeing this? Is this a cause for concern? >> >> >> I think it is quite possible that only three of them ever get used >> (which would mean that you actually >> don''t need the others). They are swapped by lighttpd - when one is >> busy, the other gets used etc. and I guess >> the first and the second one are both free when the third one is >> used, so they just get used instead of the rest. >> In general 10 FCGIs is for, like, er. a very very big site. >> >> As for the ones that have high memory utilization, check if you are >> running them in ''development''. It is notoriously leaky on FCGI. If >> even that is OK, check your actions and the parameter in your >> dispatch.fcgi (it doesn''t force garbage collection by default). > > > Just to clarify (and expand upon) this last note: FCGI in development > mode leaks Proc objects in 0.13.1. This _only_ happens in development > mode. Also, this leak has been fixed in the latest Rails and should > no longer be an issue as of 0.14. > > - Jamis > > _______________________________________________ > Rails mailing list > Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org > http://lists.rubyonrails.org/mailman/listinfo/rails
On Oct 17, 2005, at 11:53 AM, Jos wrote:> .14?! Not 1.0? > > ;):) 0.14 == 1.0 RC1 - Jamis> > Jamis Buck wrote: > > >> On Oct 16, 2005, at 9:01 PM, Julian ''Julik'' Tarkhanov wrote: >> >> >>> On 17-okt-2005, at 1:23, Tyler Kovacs wrote: >>> >>> >>>> We''re running rails + lighttpd at zvents.com. We''re starting 10 >>>> dispatch.fcgi processes when lighttpd is launched. At first, >>>> all 10 >>>> processes have approximately the same memory utilization >>>> (approx 33 MB >>>> each). After a day or so, I notice that 1 or 2 processes have high >>>> memory utilization (approx 100 MB) but the rest are still at 33MB. >>>> See the output of top (ordered by memory utilization) below. This >>>> makes me wonder if only 2 of the 10 dispatch.fcgis are actually in >>>> use. Is anyone else seeing this? Is this a cause for concern? >>>> >>> >>> >>> I think it is quite possible that only three of them ever get >>> used (which would mean that you actually >>> don''t need the others). They are swapped by lighttpd - when one >>> is busy, the other gets used etc. and I guess >>> the first and the second one are both free when the third one is >>> used, so they just get used instead of the rest. >>> In general 10 FCGIs is for, like, er. a very very big site. >>> >>> As for the ones that have high memory utilization, check if you >>> are running them in ''development''. It is notoriously leaky on >>> FCGI. If even that is OK, check your actions and the parameter >>> in your dispatch.fcgi (it doesn''t force garbage collection by >>> default). >>> >> >> >> Just to clarify (and expand upon) this last note: FCGI in >> development mode leaks Proc objects in 0.13.1. This _only_ >> happens in development mode. Also, this leak has been fixed in >> the latest Rails and should no longer be an issue as of 0.14. >> >> - Jamis >> >> _______________________________________________ >> Rails mailing list >> Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org >> http://lists.rubyonrails.org/mailman/listinfo/rails >> > > > _______________________________________________ > Rails mailing list > Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org > http://lists.rubyonrails.org/mailman/listinfo/rails >