DK
2011-Jul-07 14:59 UTC
Question on memory usage, garbage collector, ''top'' stats on linux
Hi all, I am trying to tune and troubleshoot an application which runs a background (delayed_job) process involving the import and processing of large csv files into the db and then processing of that data. When I run the process (here Ubuntu 10.04) and monitor ''top'' in the terminal, I see that ''Mem: used'' starts at maybe 500000k and then ups through the process to upwards towards 4gb (which is the system total memory). However, if I look at %MEM in the processes, I see that ruby is using maybe 10% of total memory. Also I do not see processes adding up %MEM that comes anywhere close to ''Mem: used'' at the top summary. So my questions are: (1) do I need to be concerned that used memory is approaching max and seems to be caused by running my application (no other apps or processes are running on the server that I have started). (2) Does used memory (Mem: used) mean that it can be re-allocated, or that it is locked? (I believe on the stats on the mac differentiate more between active, inactive and used, but here on linux there is just ''free'' and ''used'', and I am assuming used contains both inactive and used corresponding to the mac buckets. (3) Should I be looking at manually running the ruby garbage collector manually to mitigate such excessive memory balooning (while the actual current ruby processes are not using actively nearly so much memory)? Example (linux) towards end of large background process run: top - 13:24:52 up 5:19, 3 users, load average: 1.21, 1.42, 1.35 Tasks: 89 total, 2 running, 87 sleeping, 0 stopped, 0 zombie Cpu(s): 99.3%us, 0.7%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 3994516k total, 3809772k used, 184744k free, 119424k buffers Swap: 905208k total, 0k used, 905208k free, 3176008k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 6291 webappus 20 0 274m 151m 3276 S 0.3 3.9 8:27.12 ruby 6218 webappus 20 0 206m 82m 3544 R 98.9 2.1 0:48.81 ruby 6208 webappus 20 0 179m 59m 4788 S 0.0 1.5 0:07.50 ruby 6295 postgres 20 0 102m 32m 28m S 0.0 0.8 17:54.62 postgres 1034 postgres 20 0 98.7m 26m 25m S 0.0 0.7 0:23.67 postgres 843 mysql 20 0 174m 26m 6648 S 0.0 0.7 0:31.82 mysqld 6222 postgres 20 0 107m 19m 11m S 0.0 0.5 0:00.61 postgres 6158 root 20 0 42668 8684 2344 S 0.0 0.2 0:02.48 ruby 907 postgres 20 0 98.6m 6680 5528 S 0.0 0.2 0:13.14 postgres -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
Frederick Cheung
2011-Jul-07 21:24 UTC
Re: Question on memory usage, garbage collector, ''top'' stats on linux
On Jul 7, 3:59 pm, DK <dk.k...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> Hi all, > I am trying to tune and troubleshoot an application which runs a background > (delayed_job) process involving the import and processing of large csv files > into the db and then processing of that data. When I run the process (here > Ubuntu 10.04) and monitor ''top'' in the terminal, I see that ''Mem: used'' > starts at maybe 500000k and then ups through the process to upwards towards > 4gb (which is the system total memory). However, if I look at %MEM in the > processes, I see that ruby is using maybe 10% of total memory. Also I do not > see processes adding up %MEM that comes anywhere close to ''Mem: used'' at the > top summary. > > So my questions are: > (1) do I need to be concerned that used memory is approaching max and seems > to be caused by running my application (no other apps or processes are > running on the server that I have started). > > (2) Does used memory (Mem: used) mean that it can be re-allocated, or that > it is locked? (I believe on the stats on the mac differentiate more between > active, inactive and used, but here on linux there is just ''free'' and > ''used'', and I am assuming used contains both inactive and used corresponding > to the mac buckets. >It does include stuff that is free-able. In a nutshell, if the kernel has read some pages then even after those pages are no longer in use it will keep them around on the off chance that they''re used again (see the large Swap: cached number). You might also find the output of free -m useful. It''s unlikely that you need to run the garbage collector more often - if anything the out of the box configuration runs it too often. Fred> (3) Should I be looking at manually running the ruby garbage collector > manually to mitigate such excessive memory balooning (while the actual > current ruby processes are not using actively nearly so much memory)? > > Example (linux) towards end of large background process run: > top - 13:24:52 up 5:19, 3 users, load average: 1.21, 1.42, 1.35 > Tasks: 89 total, 2 running, 87 sleeping, 0 stopped, 0 zombie > Cpu(s): 99.3%us, 0.7%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, > 0.0%st > Mem: 3994516k total, 3809772k used, 184744k free, 119424k buffers > Swap: 905208k total, 0k used, 905208k free, 3176008k cached > > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ > COMMAND > 6291 webappus 20 0 274m 151m 3276 S 0.3 3.9 8:27.12 > ruby > 6218 webappus 20 0 206m 82m 3544 R 98.9 2.1 0:48.81 > ruby > 6208 webappus 20 0 179m 59m 4788 S 0.0 1.5 0:07.50 > ruby > 6295 postgres 20 0 102m 32m 28m S 0.0 0.8 17:54.62 > postgres > 1034 postgres 20 0 98.7m 26m 25m S 0.0 0.7 0:23.67 > postgres > 843 mysql 20 0 174m 26m 6648 S 0.0 0.7 0:31.82 > mysqld > 6222 postgres 20 0 107m 19m 11m S 0.0 0.5 0:00.61 > postgres > 6158 root 20 0 42668 8684 2344 S 0.0 0.2 0:02.48 > ruby > 907 postgres 20 0 98.6m 6680 5528 S 0.0 0.2 0:13.14 postgres-- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
Colin Law
2011-Jul-07 21:37 UTC
Re: Question on memory usage, garbage collector, ''top'' stats on linux
On 7 July 2011 15:59, DK <dk.kahn-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> Hi all, > I am trying to tune and troubleshoot an application which runs a background > (delayed_job) process involving the import and processing of large csv files > into the db and then processing of that data. When I run the process (here > Ubuntu 10.04) and monitor ''top'' in the terminal, I see that ''Mem: used'' > starts at maybe 500000k and then ups through the process to upwards towards > 4gb (which is the system total memory). However, if I look at %MEM in the > processes, I see that ruby is using maybe 10% of total memory. Also I do not > see processes adding up %MEM that comes anywhere close to ''Mem: used'' at the > top summary.My understanding is that the amount of memory that can be made available if required is the sum of free, buffered and cached. Though possibly this is a gross over simplification. Colin -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
DK
2011-Jul-07 23:15 UTC
Re: Question on memory usage, garbage collector, ''top'' stats on linux
On 7 July 2011 16:37, Colin Law <clanlaw-gM/Ye1E23mwN+BqQ9rBEUg@public.gmane.org> wrote:> On 7 July 2011 15:59, DK <dk.kahn-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote: > > Hi all, > > I am trying to tune and troubleshoot an application which runs a > background > > (delayed_job) process involving the import and processing of large csv > files > > into the db and then processing of that data. When I run the process > (here > > Ubuntu 10.04) and monitor ''top'' in the terminal, I see that ''Mem: used'' > > starts at maybe 500000k and then ups through the process to upwards > towards > > 4gb (which is the system total memory). However, if I look at %MEM in the > > processes, I see that ruby is using maybe 10% of total memory. Also I do > not > > see processes adding up %MEM that comes anywhere close to ''Mem: used'' at > the > > top summary. > > My understanding is that the amount of memory that can be made > available if required is the sum of free, buffered and cached. Though > possibly this is a gross over simplification. >Thanks guys. What is interesting and seems to support Fredericks statement about the garbage collector is that when I did put in a ''GC.start'' call after each major step in the process it did not seem to free up any memory, in fact the memory stats looked a bit worse. As far as cached, I see in top: Mem: 3994516k total, 3830048k used, 164468k free, 119540k buffers Swap: 905208k total, 0k used, 905208k free, 3207936k cached But the ''cached'' seems to fall under Swap. Do you think this is relating to Mem and not the Swap file? Would make sense as would leave ~600-700 mb as the actual memory being used which seems to correspond to what I see being taken by the actual processes.> > Colin > > -- > You received this message because you are subscribed to the Google Groups > "Ruby on Rails: Talk" group. > To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > To unsubscribe from this group, send email to > rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > For more options, visit this group at > http://groups.google.com/group/rubyonrails-talk?hl=en. > >-- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
Andrew Skegg
2011-Jul-10 23:56 UTC
Re: Question on memory usage, garbage collector, ''top'' stats on linux
DK <dk.kahn@...> writes:> > As far as cached, I see in top: > > > Mem: 3994516k total, 3830048k used, 164468k free, 119540k buffers > Swap: 905208k total, 0k used, 905208k free, 3207936k cached >Being simplistic, I would not worry about it unless the swap file is actually being used (which would indicate you''re running out of actual RAM). -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.