I am noticing considerably longer first page load times on most of my rails apps running Rails (1.1.2) off Mongrel (0.13) on linux. After the initial request everything seems to run OK. Has anybody else noticed this and is there anyway to compensate. Also following the instructions for working with unicode from the rails wiki, the number of pages served drops noticeably. Are there any sites on how to make rails more performant? Thanks in advance, Michael -- Posted via http://www.ruby-forum.com/.
Jon Gretar Borgthorsson
2006-May-15 04:33 UTC
[Rails] Mongrel / RoR first page load times.
I actually have the same first load problem when simply using the built in webrick server. I''m wondering if it''s loading some initial process to memory or something. On 5/14/06, Michael <rails@eight-lines.com> wrote:> I am noticing considerably longer first page load times on most of my > rails apps running Rails (1.1.2) off Mongrel (0.13) on linux. After the > initial request everything seems to run OK. Has anybody else noticed > this and is there anyway to compensate. > > Also following the instructions for working with unicode from the rails > wiki, the number of pages served drops noticeably. Are there any sites > on how to make rails more performant? > > Thanks in advance, > > Michael > > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > Rails mailing list > Rails@lists.rubyonrails.org > http://lists.rubyonrails.org/mailman/listinfo/rails >-- -------------- Jon Gretar Borgthorsson http://www.jongretar.net/
On Mon, 2006-05-15 at 04:52 +0200, Michael wrote:> I am noticing considerably longer first page load times on most of my > rails apps running Rails (1.1.2) off Mongrel (0.13) on linux. After the > initial request everything seems to run OK. Has anybody else noticed > this and is there anyway to compensate. >This warm-up perioud happens with just about any Rails application (and frankly, *any* application in *any* language) since it has to load and configure quite a bit of machinery on the first load. This is more pronounced if you''re loading large numbers of objects with your AR usage. Also, you should look in your production.log file to see what kind of timings you are getting. Basically, you are following a red-herring by saying "mongrel" is causing slow page load. You need to look at your Rails application and start analyzing how you''ve implemented it.> Also following the instructions for working with unicode from the rails > wiki, the number of pages served drops noticeably. Are there any sites > on how to make rails more performant?Read about the following topics: * Page caching * Fragment caching * Reverse proxy caching * Hand writing SQL for extra speed That will be the first start. After that you want to look at things you can do to just not use Rails. For example, if you''re use IO.popen for anything then you''ll want to convert that to a DRb server that doesn''t constantly open processes. Another thing I would need to point out is that you want to focus your energy on measurable goals. Instead of saying, "MY GOD, THIS IS SO SLOW!" you should try to determine a measurable *target* speed you need for your application, and then do the minimal effort to reach that goal. Basically, you have better things to do than chase some imaginary performance need. Many times just a few simple fixes here and there will get you a long way. -- Zed A. Shaw http://www.zedshaw.com/ http://mongrel.rubyforge.org/
Thanks Zed for the info.> Basically, you are following a red-herring by saying "mongrel" is > causing slow page load. You need to look at your Rails application and > start analyzing how you''ve implemented it.I was not trying to imply that Mongrel was the cause (subject a little misleading I guess). Rails deployment can be a real pain in the ass; Mongrel simplifies this dramatically, so its all I am using at the moment.> This warm-up perioud happens with just about any Rails application (and > frankly, *any* application in *any* language) since it has to load and > configure quite a bit of machinery on the first load.You are right, noticed the lag on less frequented rails sites. With regards to my app, I have quite a few large arrays that get required with enviroment.rb and that''s about it (pretty simple), I thought this was a one time affair? The lag is like 15-20s for the first page then after that its "instant". Hardware constraints are probably also a factor. Is there any way to keep the rails app loaded other than possibly polling the app with a cron job? Will keep on looking - thanks for the help -- Posted via http://www.ruby-forum.com/.
> The lag is like 15-20s for the first page then after that its "instant". > Hardware constraints are probably also a factor. Is there any way to > keep the rails app loaded other than possibly polling the app with a > cron job?that sounds like swapping from disk rather than a startup lag -- even servlet containers (sometimes) start quicker than that. There have been various posts and probably mentions on the wiki and blogs about pinging a site to keep it in memory... look esp for people talking about keeping apps in memory at shared host ISPs like Dreamhost and such, there are already written little one-liner scripts to do this if you don''t feel like writing your own.
i have been plaugued by the same issues with fcgi and mongrel. The first startup times are really bad, then every request after that is blazing fast. I am still yet to find any real solution to this other than cronning a wget to my site every 30 minutes, which seems really silly. adam On 5/15/06, Todd Grimason <tdgrmsn@gmail.com> wrote:> > > The lag is like 15-20s for the first page then after that its "instant". > > Hardware constraints are probably also a factor. Is there any way to > > keep the rails app loaded other than possibly polling the app with a > > cron job? > > that sounds like swapping from disk rather than a startup lag -- even > servlet containers (sometimes) start quicker than that. There have > been various posts and probably mentions on the wiki and blogs about > pinging a site to keep it in memory... look esp for people talking > about keeping apps in memory at shared host ISPs like Dreamhost and > such, there are already written little one-liner scripts to do this if > you don''t feel like writing your own. > _______________________________________________ > Rails mailing list > Rails@lists.rubyonrails.org > http://lists.rubyonrails.org/mailman/listinfo/rails >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://wrath.rubyonrails.org/pipermail/rails/attachments/20060516/94c391a1/attachment.html
In the latest version of fcgi you can set a number of processes to stay alive even if the others timeout. It is for just this type of situation. Charlie Bowman http://www.recentrambles.com On Tue, 2006-05-16 at 08:44 -0400, Adam Denenberg wrote:> i have been plaugued by the same issues with fcgi and mongrel. The > first startup times are really bad, then every request after that is > blazing fast. I am still yet to find any real solution to this other > than cronning a wget to my site every 30 minutes, which seems really > silly. > > adam > > > On 5/15/06, Todd Grimason <tdgrmsn@gmail.com> wrote: > > > The lag is like 15-20s for the first page then after that > its "instant". > > Hardware constraints are probably also a factor. Is there > any way to > > keep the rails app loaded other than possibly polling the > app with a > > cron job? > > that sounds like swapping from disk rather than a startup lag > -- even > servlet containers (sometimes) start quicker than that. There > have > been various posts and probably mentions on the wiki and blogs > about > pinging a site to keep it in memory... look esp for people > talking > about keeping apps in memory at shared host ISPs like > Dreamhost and > such, there are already written little one-liner scripts to do > this if > you don''t feel like writing your own. > _______________________________________________ > Rails mailing list > Rails@lists.rubyonrails.org > http://lists.rubyonrails.org/mailman/listinfo/rails > > > > _______________________________________________ > Rails mailing list > Rails@lists.rubyonrails.org > http://lists.rubyonrails.org/mailman/listinfo/rails-------------- next part -------------- An HTML attachment was scrubbed... URL: http://wrath.rubyonrails.org/pipermail/rails/attachments/20060516/12487de5/attachment.html
how would you handle this with mongrel ? On 5/16/06, Charlie Bowman <charlie@castlebranch.com> wrote:> > In the latest version of fcgi you can set a number of processes to stay > alive even if the others timeout. It is for just this type of situation. > > Charlie Bowman > http://www.recentrambles.com > > > > On Tue, 2006-05-16 at 08:44 -0400, Adam Denenberg wrote: > > i have been plaugued by the same issues with fcgi and mongrel. The first > startup times are really bad, then every request after that is blazing > fast. I am still yet to find any real solution to this other than cronning > a wget to my site every 30 minutes, which seems really silly. > > adam > > On 5/15/06, *Todd Grimason* <tdgrmsn@gmail.com> wrote: > > > The lag is like 15-20s for the first page then after that its "instant". > > Hardware constraints are probably also a factor. Is there any way to > > keep the rails app loaded other than possibly polling the app with a > > cron job? > > that sounds like swapping from disk rather than a startup lag -- even > servlet containers (sometimes) start quicker than that. There have > been various posts and probably mentions on the wiki and blogs about > pinging a site to keep it in memory... look esp for people talking > about keeping apps in memory at shared host ISPs like Dreamhost and > such, there are already written little one-liner scripts to do this if > you don''t feel like writing your own. > _______________________________________________ > Rails mailing list > Rails@lists.rubyonrails.org > http://lists.rubyonrails.org/mailman/listinfo/rails > > > _______________________________________________Rails mailing listRails@lists.rubyonrails.orghttp://lists.rubyonrails.org/mailman/listinfo/rails > > > > _______________________________________________ > Rails mailing list > Rails@lists.rubyonrails.org > http://lists.rubyonrails.org/mailman/listinfo/rails > > >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://wrath.rubyonrails.org/pipermail/rails/attachments/20060516/85fbaaac/attachment.html
Mongrel is a persistant processes so ''stay alive'' is a moot point. However your issue of the slow first request sounds more like it''s either the very initial startup overhead of mongrel/rails (like if you were to deploy a new version of your app to a server) or you''re running it in a low memory environment (like a zen or uml virtual server) and it''s been swapped to disk and then pulled back out of swap on the initial request. Paul. On 16/05/06, Adam Denenberg <straightflush@gmail.com> wrote:> > how would you handle this with mongrel ? > > > > On 5/16/06, Charlie Bowman <charlie@castlebranch.com> wrote: > > > > In the latest version of fcgi you can set a number of processes to stay > > alive even if the others timeout. It is for just this type of situation. > > > > Charlie Bowman > > http://www.recentrambles.com > > >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://wrath.rubyonrails.org/pipermail/rails/attachments/20060516/3a778ea6/attachment.html
If your problem is that your process (or your entire virtual machine) is being swapped out, then solving it by doing page gets every N seconds is almost certainly wrong in a shared hosting environment. You''re probably going to need to solve it by stepping up to a more resource-friendly (and presumably higher cost) environment. Forcing the host to keep your stuff hot means some other customer is suffering even more. - James Moore
This is being run on a server I have in its entirety (not a VPS). I only run apache, rails, postgres and some mail server applications on the server, so resources are definitely not the issue. Just wondering how to better handle this once mogrel is started and nobody hits the site for say, an hour. Or should this not happen at all with mongrel once the first request comes in ? adam On 5/16/06, James Moore <banshee@banshee.com> wrote:> > If your problem is that your process (or your entire virtual machine) is > being swapped out, then solving it by doing page gets every N seconds is > almost certainly wrong in a shared hosting environment. You''re probably > going to need to solve it by stepping up to a more resource-friendly (and > presumably higher cost) environment. Forcing the host to keep your stuff > hot means some other customer is suffering even more. > > - James Moore > > _______________________________________________ > Rails mailing list > Rails@lists.rubyonrails.org > http://lists.rubyonrails.org/mailman/listinfo/rails >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://wrath.rubyonrails.org/pipermail/rails/attachments/20060516/195db82b/attachment.html
On Tue, 2006-05-16 at 14:55 -0400, Adam Denenberg wrote:> This is being run on a server I have in its entirety (not a VPS). I > only run apache, rails, postgres and some mail server applications on > the server, so resources are definitely not the issue. Just wondering > how to better handle this once mogrel is started and nobody hits the > site for say, an hour. Or should this not happen at all with mongrel > once the first request comes in ?Adam, since nobody really has access to your server you''re going to have to do more investigation into what can cause it. Typically people approach problems like this with the attitude of, "Well, *my* code/server/database/sql/hardware is perfect, so it *must* be mongrel/rails/mysql." Not saying this is you, but asking the list without providing more evidence about your investigation so far tends to imply that. A better way to figure out the cause is to just follow the scientific method. It''s pretty simple, and don''t really want to patronize you, so here''s just how I do it: 1) You''re observing a set of strange behaviors. Do an initial investigation trying to collect some data. For example, finding out how long the server has to be idle before it has this pause would be a good clue. 2) Develop a set of hypothesis that might be causing it. Notice I''m saying "might" not "does". This can be anything from "Mongrel is a piece of crap" to "I''ve got my SQL queries pulling an too much data." 3) Pick a hypothesis and then develop a *measurable* test that you can run to confirm this hypothesis. Notice I''m not saying "prove". This isn''t about being right or wrong but solving a problem. Also having the test be measurable is really important since it keeps you honest and may show you more evidence. 4) Once you think you''ve got the answer, conduct another test that either invalidates the other hypotheses, or try to force your proposed cause. What you''re doing here is either ruling out the other possible causes--giving your best guess better probability--or you''re actually forcing it to happen. 5) Keep repeating 2-4 until you hit on a cause. Keep in mind sometimes it''s three or four things interacting together that causes you problems. 6) NOW implement a fix. As you go along and gather more evidence, keep notes and when you get stuck, come back to the list and lay out your process so far. It might be that you don''t know some piece of information, but the second someone else sees your data and tests so far they know the problem right away. Hope that helps. Not patronizing you, just trying to give you some tools to figure it out for yourself since you''re kind of stuck solving it independently. -- Zed A. Shaw http://www.zedshaw.com/ http://mongrel.rubyforge.org/
zed, I am definitely not blaming mongrel (I think it rocks actually) , and I dont disagree the amount of hard data I have provided has been pretty limited, if not at all. I guess maybe I was just under the assumption that this was a common issue that someone already figured out, however I see this issue pop up all the time with no real resolution. It seems that with about an hour or so of inactivity the first request to a rails app is fairly slow, and kind of hangs on the "Waiting for site domain.com..." in the browser. Once the first request comes in, future requests are blazing fast. the thing is, this has happend with both FCGI and Mongrel, so I was under the assumption this is some sort of Rails issue, or there was some configuration that was known to cause this. Maybe rails keeps things in memory for a configurable amount of time which explains the quick speeds after a first request? these are the things I thought were the path to the answer. Anyway thanks for the troubleshooting tips, I will try some of them out and see what I come up with for some more hard data. adam On 5/16/06, Zed Shaw <zedshaw@zedshaw.com> wrote:> > On Tue, 2006-05-16 at 14:55 -0400, Adam Denenberg wrote: > > This is being run on a server I have in its entirety (not a VPS). I > > only run apache, rails, postgres and some mail server applications on > > the server, so resources are definitely not the issue. Just wondering > > how to better handle this once mogrel is started and nobody hits the > > site for say, an hour. Or should this not happen at all with mongrel > > once the first request comes in ? > > Adam, since nobody really has access to your server you''re going to have > to do more investigation into what can cause it. Typically people > approach problems like this with the attitude of, "Well, *my* > code/server/database/sql/hardware is perfect, so it *must* be > mongrel/rails/mysql." Not saying this is you, but asking the list > without providing more evidence about your investigation so far tends to > imply that. > > A better way to figure out the cause is to just follow the scientific > method. It''s pretty simple, and don''t really want to patronize you, so > here''s just how I do it: > > 1) You''re observing a set of strange behaviors. Do an initial > investigation trying to collect some data. For example, finding out how > long the server has to be idle before it has this pause would be a good > clue. > > 2) Develop a set of hypothesis that might be causing it. Notice I''m > saying "might" not "does". This can be anything from "Mongrel is a > piece of crap" to "I''ve got my SQL queries pulling an too much data." > > 3) Pick a hypothesis and then develop a *measurable* test that you can > run to confirm this hypothesis. Notice I''m not saying "prove". This > isn''t about being right or wrong but solving a problem. Also having the > test be measurable is really important since it keeps you honest and may > show you more evidence. > > 4) Once you think you''ve got the answer, conduct another test that > either invalidates the other hypotheses, or try to force your proposed > cause. What you''re doing here is either ruling out the other possible > causes--giving your best guess better probability--or you''re actually > forcing it to happen. > > 5) Keep repeating 2-4 until you hit on a cause. Keep in mind sometimes > it''s three or four things interacting together that causes you problems. > > 6) NOW implement a fix. > > As you go along and gather more evidence, keep notes and when you get > stuck, come back to the list and lay out your process so far. It might > be that you don''t know some piece of information, but the second someone > else sees your data and tests so far they know the problem right away. > > Hope that helps. Not patronizing you, just trying to give you some > tools to figure it out for yourself since you''re kind of stuck solving > it independently. > > > -- > Zed A. Shaw > http://www.zedshaw.com/ > http://mongrel.rubyforge.org/ > > > _______________________________________________ > Rails mailing list > Rails@lists.rubyonrails.org > http://lists.rubyonrails.org/mailman/listinfo/rails >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://wrath.rubyonrails.org/pipermail/rails/attachments/20060516/1bd64efb/attachment-0001.html
Adam Denenberg wrote:> It seems that with about an hour or so of inactivity the first request > to a rails app is fairly slow, and kind of hangs on the "Waiting for > site domain.com..." in the browser. Once the first request comes in, > future requests are blazing fast. the thing is, this has happend with > both FCGI and Mongrel, so I was under the assumption this is some sort > of Rails issue, or there was some configuration that was known to cause > this. Maybe rails keeps things in memory for a configurable amount of > time which explains the quick speeds after a first request? these are > the things I thought were the path to the answer.I saw exactly this behavior with using lighty+fcgi on my leased box. I had overextended it the lighty+fcgi processes were completely swapped out. cat /proc/meminfo will give you a general idea of what is going on. You can use top (make sure to turn the swap field on by pressing "f" while its running then "p" to choose the swap field). This should give you an idea of what, if anything, is getting swapped out. Austin
I guess you could be right. this is what I saw. SWAP PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 0 4294 root 17 0 1120 1120 772 R 13.3 0.1 0:07.23 top 416 20228 root 17 0 34740 33m 33m S 9.3 3.4 0:43.81mongrel_rails On 5/17/06, Austin Godber <godber@uberhip.com> wrote:> > Adam Denenberg wrote: > > It seems that with about an hour or so of inactivity the first request > > to a rails app is fairly slow, and kind of hangs on the "Waiting for > > site domain.com..." in the browser. Once the first request comes in, > > future requests are blazing fast. the thing is, this has happend with > > both FCGI and Mongrel, so I was under the assumption this is some sort > > of Rails issue, or there was some configuration that was known to cause > > this. Maybe rails keeps things in memory for a configurable amount of > > time which explains the quick speeds after a first request? these are > > the things I thought were the path to the answer. > > I saw exactly this behavior with using lighty+fcgi on my leased box. I > had overextended it the lighty+fcgi processes were completely swapped out. > > cat /proc/meminfo > > will give you a general idea of what is going on. You can use top (make > sure to turn the swap field on by pressing "f" while its running then > "p" to choose the swap field). This should give you an idea of what, if > anything, is getting swapped out. > > Austin > _______________________________________________ > Rails mailing list > Rails@lists.rubyonrails.org > http://lists.rubyonrails.org/mailman/listinfo/rails >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://wrath.rubyonrails.org/pipermail/rails/attachments/20060517/df42e7f9/attachment.html
Adam Denenberg wrote:> I guess you could be right. this is what I saw. > > SWAP PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > 0 4294 root 17 0 1120 1120 772 R 13.3 0.1 0:07.23 top > 416 20228 root 17 0 34740 33m 33m S 9.3 3.4 0:43.81 mongrel_railsHmm ... not necessarily. Whats the overall swap and memory usage? See the top few lines of top ... or cat /proc/meminfo Chances are you will see at least a little bit swapped out on several processes (if your consuming a good deal of memory). The fact that the number is not zero should not be startling. 416 bytes out of 33m is not much. Then again ... was this after the app sat untouched for a few hours? Or right after servicing some requests? One thing I really don''t have a good feel for is how big mongrel (or other rails containers) gets for different size rails apps. Most of my apps are fairly simple. If anyone has a good feel for that, please share. Austin
On May 17, 2006, at 10:01 PM, Austin Godber wrote:> Adam Denenberg wrote: >> I guess you could be right. this is what I saw. >> >> SWAP PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ >> COMMAND >> 0 4294 root 17 0 1120 1120 772 R 13.3 0.1 0:07.23 top >> 416 20228 root 17 0 34740 33m 33m S 9.3 3.4 0:43.81 >> mongrel_rails > > Hmm ... not necessarily. Whats the overall swap and memory usage? > See > the top few lines of top ... or cat /proc/meminfo > > Chances are you will see at least a little bit swapped out on several > processes (if your consuming a good deal of memory). The fact that > the > number is not zero should not be startling. 416 bytes out of 33m > is not > much. Then again ... was this after the app sat untouched for a few > hours? Or right after servicing some requests? > > One thing I really don''t have a good feel for is how big mongrel (or > other rails containers) gets for different size rails apps. Most > of my > apps are fairly simple. If anyone has a good feel for that, please > share.The way I read that is: 33M+ total process size (VIRT), 33M in RAM (RES), and 33M available for sharing with other processes (SHR). If that''s correct, then you could run several and not use any more memory. Is that possible? -- -- Tom Mornini
here is a copy of the /proc/meminfo. There are a couple of java apps
running that could be using a decent amount of mem that could be accounting
for the intial swapping. That data i sent in my previous email was after
the app had no requests for quite a few hours but the mongrels have been up
for a few days.
total: used: free: shared: buffers: cached:
Mem: 1047658496 979578880 68079616 0 39374848 610492416
Swap: 2097434624 845279232 1252155392
MemTotal: 1023104 kB
MemFree: 66484 kB
MemShared: 0 kB
Buffers: 38452 kB
Cached: 272152 kB
SwapCached: 324032 kB
Active: 574344 kB
Inact_dirty: 207740 kB
Inact_clean: 77208 kB
Inact_target: 171856 kB
HighTotal: 122816 kB
HighFree: 960 kB
LowTotal: 900288 kB
LowFree: 65524 kB
SwapTotal: 2048276 kB
SwapFree: 1222808 kB
Committed_AS: 1379188 kB
adam
On 5/18/06, Tom Mornini <tmornini@infomania.com>
wrote:>
> On May 17, 2006, at 10:01 PM, Austin Godber wrote:
>
> > Adam Denenberg wrote:
> >> I guess you could be right. this is what I saw.
> >>
> >> SWAP PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+
> >> COMMAND
> >> 0 4294 root 17 0 1120 1120 772 R 13.3 0.1 0:07.23
top
> >> 416 20228 root 17 0 34740 33m 33m S 9.3 3.4 0:43.81
> >> mongrel_rails
> >
> > Hmm ... not necessarily. Whats the overall swap and memory usage?
> > See
> > the top few lines of top ... or cat /proc/meminfo
> >
> > Chances are you will see at least a little bit swapped out on several
> > processes (if your consuming a good deal of memory). The fact that
> > the
> > number is not zero should not be startling. 416 bytes out of 33m
> > is not
> > much. Then again ... was this after the app sat untouched for a few
> > hours? Or right after servicing some requests?
> >
> > One thing I really don''t have a good feel for is how big
mongrel (or
> > other rails containers) gets for different size rails apps. Most
> > of my
> > apps are fairly simple. If anyone has a good feel for that, please
> > share.
>
> The way I read that is:
>
> 33M+ total process size (VIRT), 33M in RAM (RES), and 33M available for
> sharing with other processes (SHR).
>
> If that''s correct, then you could run several and not use any more
> memory.
>
> Is that possible?
>
> --
> -- Tom Mornini
>
> _______________________________________________
> Rails mailing list
> Rails@lists.rubyonrails.org
> http://lists.rubyonrails.org/mailman/listinfo/rails
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://wrath.rubyonrails.org/pipermail/rails/attachments/20060518/749484a5/attachment-0001.html
(Professional sysadmins laugh at me, so take this with a grain of salt...) You''ve got serious memory problems. You''ve more than filled your memory - the numbers for swap are horrible. Ideally you want SwapFree and SwapTotal to be the same (as in swapping == pain). You certainly want to slap in another gig of memory, maybe more, and then keep an eye on things. Why do you say "initial" swapping? The system''s running at max memory right now, no? MemFree is essentially zero. - James Moore ________________________________________ From: rails-bounces@lists.rubyonrails.org [mailto:rails-bounces@lists.rubyonrails.org] On Behalf Of Adam Denenberg Sent: Thursday, May 18, 2006 4:36 AM To: rails@lists.rubyonrails.org Subject: Re: [Rails] Re: Mongrel / RoR first page load times. here is a copy of the /proc/meminfo.? There are a couple of java apps running that could be using a decent amount of mem that could be accounting for the intial swapping.? That data i sent in my previous email was after the app had no requests for quite a few hours but the mongrels have been up for a few days. ??????? total:??? used:??? free:? shared: buffers:? cached: Mem:? 1047658496 979578880 68079616??????? 0 39374848 610492416 Swap: 2097434624 845279232 1252155392 MemTotal:????? 1023104 kB MemFree:???????? 66484 kB MemShared:?????????? 0 kB Buffers:???????? 38452 kB Cached:???????? 272152 kB SwapCached:???? 324032 kB Active:???????? 574344 kB Inact_dirty:??? 207740 kB Inact_clean:???? 77208 kB Inact_target:?? 171856 kB HighTotal:????? 122816 kB HighFree:????????? 960 kB LowTotal:?????? 900288 kB LowFree:???????? 65524 kB SwapTotal:???? 2048276 kB SwapFree:????? 1222808 kB Committed_AS:? 1379188 kB adam
this is the way it shows up on Linux systems from my understanding. Linux
sucks in all the memory into the disk cache mostly, and then distributes it
as needed. I think on almost any linux system the memory thru top appears
almost 100% utilized but thats just how the kernel manages it. I believe
thats the case here, although i am swapping more than I should be. Free -m
shows this
# free -m
total used free shared buffers
cached
Mem: 999 977 22 0 32 262
-/+ buffers/cache: 681 317
Swap: 2000 756 1244
adam
On 5/18/06, James Moore <banshee@banshee.com>
wrote:>
> (Professional sysadmins laugh at me, so take this with a grain of salt...)
>
> You''ve got serious memory problems. You''ve more than
filled your memory -
> the numbers for swap are horrible. Ideally you want SwapFree and
> SwapTotal
> to be the same (as in swapping == pain). You certainly want to slap in
> another gig of memory, maybe more, and then keep an eye on things.
>
> Why do you say "initial" swapping? The system''s running
at max memory
> right
> now, no? MemFree is essentially zero.
>
> - James Moore
> ________________________________________
> From: rails-bounces@lists.rubyonrails.org
> [mailto:rails-bounces@lists.rubyonrails.org] On Behalf Of Adam Denenberg
> Sent: Thursday, May 18, 2006 4:36 AM
> To: rails@lists.rubyonrails.org
> Subject: Re: [Rails] Re: Mongrel / RoR first page load times.
>
> here is a copy of the /proc/meminfo. There are a couple of java apps
> running that could be using a decent amount of mem that could be
> accounting
> for the intial swapping. That data i sent in my previous email was after
> the app had no requests for quite a few hours but the mongrels have been
> up
> for a few days.
>
> total: used: free: shared: buffers: cached:
> Mem: 1047658496 979578880 68079616 0 39374848 610492416
> Swap: 2097434624 845279232 1252155392
> MemTotal: 1023104 kB
> MemFree: 66484 kB
> MemShared: 0 kB
> Buffers: 38452 kB
> Cached: 272152 kB
> SwapCached: 324032 kB
> Active: 574344 kB
> Inact_dirty: 207740 kB
> Inact_clean: 77208 kB
> Inact_target: 171856 kB
> HighTotal: 122816 kB
> HighFree: 960 kB
> LowTotal: 900288 kB
> LowFree: 65524 kB
> SwapTotal: 2048276 kB
> SwapFree: 1222808 kB
> Committed_AS: 1379188 kB
>
> adam
>
> _______________________________________________
> Rails mailing list
> Rails@lists.rubyonrails.org
> http://lists.rubyonrails.org/mailman/listinfo/rails
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://wrath.rubyonrails.org/pipermail/rails/attachments/20060518/e4d16ae3/attachment.html
Linux will grab hold of *unused* memory to use for buffers and caches,
but on your system you have over 700MB in swap which isn''t a great
situation. For comparison here''s the stats on our production server:
# free -m
total used free shared buffers cached
Mem: 3963 3118 845 0 184 1695
-/+ buffers/cache: 1238 2725
Swap: 999 0 999
I''d suggest upgrading the ram on the server to 2GB as it looks like
that would solve most of the problem.
Paul.
On 18/05/06, Adam Denenberg <straightflush@gmail.com>
wrote:> this is the way it shows up on Linux systems from my understanding. Linux
> sucks in all the memory into the disk cache mostly, and then distributes it
> as needed. I think on almost any linux system the memory thru top appears
> almost 100% utilized but thats just how the kernel manages it. I believe
> thats the case here, although i am swapping more than I should be. Free -m
> shows this
>
> # free -m
>
> total used free shared buffers
> cached
> Mem: 999 977 22 0 32 262
> -/+ buffers/cache: 681 317
> Swap: 2000 756 1244
>
>
> adam
>
>
> On 5/18/06, James Moore <banshee@banshee.com> wrote:
> > (Professional sysadmins laugh at me, so take this with a grain of
salt...)
> >
> > You''ve got serious memory problems. You''ve more
than filled your memory -
> > the numbers for swap are horrible. Ideally you want SwapFree and
> SwapTotal
> > to be the same (as in swapping == pain). You certainly want to slap
in
> > another gig of memory, maybe more, and then keep an eye on things.
> >
> > Why do you say "initial" swapping? The system''s
running at max memory
> right
> > now, no? MemFree is essentially zero.
> >
> > - James Moore
> > ________________________________________
> > From: rails-bounces@lists.rubyonrails.org
> > [mailto:rails-bounces@lists.rubyonrails.org] On Behalf Of
> Adam Denenberg
> > Sent: Thursday, May 18, 2006 4:36 AM
> > To: rails@lists.rubyonrails.org
> > Subject: Re: [Rails] Re: Mongrel / RoR first page load times.
> >
> > here is a copy of the /proc/meminfo. There are a couple of java apps
> > running that could be using a decent amount of mem that could be
> accounting
> > for the intial swapping. That data i sent in my previous email was
after
> > the app had no requests for quite a few hours but the mongrels have
been
> up
> > for a few days.
> >
> > total: used: free: shared: buffers: cached:
> > Mem: 1047658496 979578880 68079616 0 39374848 610492416
> > Swap: 2097434624 845279232 1252155392
> > MemTotal: 1023104 kB
> > MemFree: 66484 kB
> > MemShared: 0 kB
> > Buffers: 38452 kB
> > Cached: 272152 kB
> > SwapCached: 324032 kB
> > Active: 574344 kB
> > Inact_dirty: 207740 kB
> > Inact_clean: 77208 kB
> > Inact_target: 171856 kB
> > HighTotal: 122816 kB
> > HighFree: 960 kB
> > LowTotal: 900288 kB
> > LowFree: 65524 kB
> > SwapTotal: 2048276 kB
> > SwapFree: 1222808 kB
> > Committed_AS: 1379188 kB
> >
> > adam
> >
> > _______________________________________________
> > Rails mailing list
> > Rails@lists.rubyonrails.org
> > http://lists.rubyonrails.org/mailman/listinfo/rails
> >
>
>
> _______________________________________________
> Rails mailing list
> Rails@lists.rubyonrails.org
> http://lists.rubyonrails.org/mailman/listinfo/rails
>
>
>
Yeah ... ouch. 800MB swapped on 1GB of real memory? Linux will definately aggressively keep stuff in REAL memory so its not unusual to see used memory at about 100% of total real memory, but the same is not true of swap. But this is precisely the same scenario I had about 3 weeks ago. I had about 256MB swapped on 512MB of real ram ... anytime I would come back to the site it would take a while to load then be snappy. Then two days later as the busier services pushed lightty+fcgi into swap I was back in Suckville. I bumped up to 1.5GB and was all better. Get some more RAM or migrate some services elsewhere. This is the point where I start wishing everything was a virtual machine ... Austin Paul Wright wrote:> Linux will grab hold of *unused* memory to use for buffers and caches, > but on your system you have over 700MB in swap which isn''t a great > situation. For comparison here''s the stats on our production server: > > # free -m > total used free shared buffers cached > Mem: 3963 3118 845 0 184 1695 > -/+ buffers/cache: 1238 2725 > Swap: 999 0 999 > > I''d suggest upgrading the ram on the server to 2GB as it looks like > that would solve most of the problem. > > Paul. > > On 18/05/06, Adam Denenberg <straightflush@gmail.com> wrote: >> this is the way it shows up on Linux systems from my understanding. >> Linux >> sucks in all the memory into the disk cache mostly, and then >> distributes it >> as needed. I think on almost any linux system the memory thru top >> appears >> almost 100% utilized but thats just how the kernel manages it. I believe >> thats the case here, although i am swapping more than I should be. >> Free -m >> shows this >> >> # free -m >> >> total used free shared buffers >> cached >> Mem: 999 977 22 0 32 262 >> -/+ buffers/cache: 681 317 >> Swap: 2000 756 1244 >> >> >> adam >> >> >> On 5/18/06, James Moore <banshee@banshee.com> wrote: >> > (Professional sysadmins laugh at me, so take this with a grain of >> salt...) >> > >> > You''ve got serious memory problems. You''ve more than filled your >> memory - >> > the numbers for swap are horrible. Ideally you want SwapFree and >> SwapTotal >> > to be the same (as in swapping == pain). You certainly want to slap in >> > another gig of memory, maybe more, and then keep an eye on things. >> > >> > Why do you say "initial" swapping? The system''s running at max memory >> right >> > now, no? MemFree is essentially zero. >> > >> > - James Moore >> > ________________________________________ >> > From: rails-bounces@lists.rubyonrails.org >> > [mailto:rails-bounces@lists.rubyonrails.org] On Behalf Of >> Adam Denenberg >> > Sent: Thursday, May 18, 2006 4:36 AM >> > To: rails@lists.rubyonrails.org >> > Subject: Re: [Rails] Re: Mongrel / RoR first page load times. >> > >> > here is a copy of the /proc/meminfo. There are a couple of java apps >> > running that could be using a decent amount of mem that could be >> accounting >> > for the intial swapping. That data i sent in my previous email was >> after >> > the app had no requests for quite a few hours but the mongrels have >> been >> up >> > for a few days. >> > >> > total: used: free: shared: buffers: cached: >> > Mem: 1047658496 979578880 68079616 0 39374848 610492416 >> > Swap: 2097434624 845279232 1252155392 >> > MemTotal: 1023104 kB >> > MemFree: 66484 kB >> > MemShared: 0 kB >> > Buffers: 38452 kB >> > Cached: 272152 kB >> > SwapCached: 324032 kB >> > Active: 574344 kB >> > Inact_dirty: 207740 kB >> > Inact_clean: 77208 kB >> > Inact_target: 171856 kB >> > HighTotal: 122816 kB >> > HighFree: 960 kB >> > LowTotal: 900288 kB >> > LowFree: 65524 kB >> > SwapTotal: 2048276 kB >> > SwapFree: 1222808 kB >> > Committed_AS: 1379188 kB >> > >> > adam >> > >> > _______________________________________________ >> > Rails mailing list >> > Rails@lists.rubyonrails.org >> > http://lists.rubyonrails.org/mailman/listinfo/rails >> > >> >> >> _______________________________________________ >> Rails mailing list >> Rails@lists.rubyonrails.org >> http://lists.rubyonrails.org/mailman/listinfo/rails >> >> >> > _______________________________________________ > Rails mailing list > Rails@lists.rubyonrails.org > http://lists.rubyonrails.org/mailman/listinfo/rails
Julian ''Julik'' Tarkhanov
2006-May-18 21:24 UTC
[Rails] Re: Mongrel / RoR first page load times.
On 16-mei-2006, at 14:44, Adam Denenberg wrote:> i have been plaugued by the same issues with fcgi and mongrel. The > first startup times are really bad, then every request after that > is blazing fast. I am still yet to find any real solution to this > other than cronning a wget to my site every 30 minutes, which seems > really silly.I think the only thing that would help will be to preload all of your classes when the app starts up. -- Julian ''Julik'' Tarkhanov please send all personal mail to me at julik.nl
I have seen rails training costing from 2k - 5k given by the Rails gurus. Since I am not a big fan of throwing away money and expecting to cram everything in 3 to 5 days, I am looking for someone who is willing to provide online training for newbies. You could just conduct the training twice or three times a week. One hour per session would be good enough. Just a simple webcam and the ability to ask questions and get doubts clarified would be great. Very practical and hands-on approach with tons of exercises/homework would be great. This would be a great opportunity for developers who work full time to use whatever time is left to learn Rails quickly. Plus no travel expenses and no need to use your vacation time!!! It would also be great to customize the training according to the needs of the students. Please reply to this thread so that we can find instructors as well as students who want to enroll in the course. http://www.ProblemSolvingSkill.net Hone your problem-solving skills
Julian ''Julik'' Tarkhanov
2006-May-18 21:55 UTC
[Rails] Instructor needed for Rails online training
On 18-mei-2006, at 23:46, Bala Paranj wrote:> > Please reply to this thread so that we can find instructors as well > as students who want to enroll > in the course.Please learn to not steal threads first. -- Julian ''Julik'' Tarkhanov please send all personal mail to me at julik.nl
Where is your thread buddy? Is the training already available? How can I enroll? --- Julian ''Julik'' Tarkhanov <listbox@julik.nl> wrote:> > On 18-mei-2006, at 23:46, Bala Paranj wrote: > > > > Please reply to this thread so that we can find instructors as well > > as students who want to enroll > > in the course. > > Please learn to not steal threads first. > > -- > Julian ''Julik'' Tarkhanov > please send all personal mail to > me at julik.nl > > > _______________________________________________ > Rails mailing list > Rails@lists.rubyonrails.org > http://lists.rubyonrails.org/mailman/listinfo/rails >http://www.ProblemSolvingSkill.net Hone your problem-solving skills
Bala Paranj wrote:> Where is your thread buddy? Is the training already available? How can I enroll?Julik was pointing out that you hijacked the thread "Mongrel / RoR first page load times", by replying to a message in that thread and changing the subject line. You should have posted a new message, rather than replying in an existing thread. Justin> --- Julian ''Julik'' Tarkhanov <listbox@julik.nl> wrote: > >> On 18-mei-2006, at 23:46, Bala Paranj wrote: >>> Please reply to this thread so that we can find instructors as well >>> as students who want to enroll >>> in the course. >> Please learn to not steal threads first. >> >> -- >> Julian ''Julik'' Tarkhanov >> please send all personal mail to >> me at julik.nl >> >> >> _______________________________________________ >> Rails mailing list >> Rails@lists.rubyonrails.org >> http://lists.rubyonrails.org/mailman/listinfo/rails >> > > > http://www.ProblemSolvingSkill.net > Hone your problem-solving skills > _______________________________________________ > Rails mailing list > Rails@lists.rubyonrails.org > http://lists.rubyonrails.org/mailman/listinfo/rails > >