thr3ads.net - Rails - rails and passenger memory issues [Dec 2013]

If this information is useful, please help other people find it:
Share via:

John Merlino

2013-Dec-04 01:03 UTC

rails and passenger memory issues

I''m using Rails 3, Passenger, and Postgresql. Yesterday the rails 
application was running very slow. So I rebooted the server. Less than 24 
hours later, it is consuming even more memory than before and when trying 
to load the web page, it just hangs forever. I ran passenger-memory-stats 
to inspect the issue. And once again, I notice 2 rack processes using over 
2 gigabytes of private dirty RSS:


$ rvmsudo -E /usr/local/rvm/gems/ruby-1.9.3-p327/bin/passenger-memory-stats
Use of uninitialized value $includedir in concatenation (.) or string at 
(eval 9) line 1.
Use of uninitialized value $includedir in concatenation (.) or string at 
(eval 9) line 1.

--------- Apache processes ----------
PID    PPID  VMSize    Private  Name
-------------------------------------
1079   1     145.6 MB  0.3 MB   /usr/sbin/apache2 -k start
2163   1079  160.5 MB  15.2 MB  /usr/sbin/apache2 -k start
2450   1079  145.9 MB  0.5 MB   /usr/sbin/apache2 -k start
5638   1079  161.1 MB  15.8 MB  /usr/sbin/apache2 -k start
6923   1079  145.8 MB  0.4 MB   /usr/sbin/apache2 -k start
9829   1079  160.0 MB  14.9 MB  /usr/sbin/apache2 -k start
11016  1079  160.5 MB  15.3 MB  /usr/sbin/apache2 -k start
11097  1079  160.5 MB  15.3 MB  /usr/sbin/apache2 -k start
11948  1079  145.8 MB  0.4 MB   /usr/sbin/apache2 -k start
11961  1079  145.8 MB  0.3 MB   /usr/sbin/apache2 -k start
11969  1079  145.8 MB  0.3 MB   /usr/sbin/apache2 -k start
12460  1079  145.6 MB  0.2 MB   /usr/sbin/apache2 -k start
12461  1079  145.6 MB  0.2 MB   /usr/sbin/apache2 -k start
12462  1079  145.7 MB  0.2 MB   /usr/sbin/apache2 -k start
12491  1079  145.6 MB  0.2 MB   /usr/sbin/apache2 -k start
12533  1079  157.5 MB  14.4 MB  /usr/sbin/apache2 -k start
12535  1079  159.8 MB  14.6 MB  /usr/sbin/apache2 -k start
12536  1079  157.7 MB  14.3 MB  /usr/sbin/apache2 -k start
12552  1079  145.6 MB  0.2 MB   /usr/sbin/apache2 -k start
12717  1079  145.6 MB  0.2 MB   /usr/sbin/apache2 -k start
12757  1079  150.1 MB  4.6 MB   /usr/sbin/apache2 -k start
12990  1079  145.6 MB  0.2 MB   /usr/sbin/apache2 -k start
13015  1079  150.3 MB  5.2 MB   /usr/sbin/apache2 -k start
13016  1079  159.8 MB  14.4 MB  /usr/sbin/apache2 -k start
13045  1079  147.9 MB  4.3 MB   /usr/sbin/apache2 -k start
13046  1079  145.6 MB  0.2 MB   /usr/sbin/apache2 -k start
13047  1079  152.4 MB  8.9 MB   /usr/sbin/apache2 -k start
13288  1079  145.6 MB  0.2 MB   /usr/sbin/apache2 -k start
13324  1079  152.4 MB  8.9 MB   /usr/sbin/apache2 -k start
13329  1079  145.6 MB  0.2 MB   /usr/sbin/apache2 -k start
13331  1079  145.6 MB  0.2 MB   /usr/sbin/apache2 -k start
13959  1079  145.6 MB  0.2 MB   /usr/sbin/apache2 -k start
13960  1079  145.6 MB  0.2 MB   /usr/sbin/apache2 -k start
13961  1079  149.0 MB  3.8 MB   /usr/sbin/apache2 -k start
14037  1079  145.6 MB  0.2 MB   /usr/sbin/apache2 -k start
14040  1079  145.6 MB  0.2 MB   /usr/sbin/apache2 -k start
14051  1079  145.6 MB  0.2 MB   /usr/sbin/apache2 -k start
14052  1079  145.6 MB  0.2 MB   /usr/sbin/apache2 -k start
14053  1079  145.6 MB  0.2 MB   /usr/sbin/apache2 -k start
14059  1079  145.6 MB  0.2 MB   /usr/sbin/apache2 -k start
14061  1079  145.6 MB  0.2 MB   /usr/sbin/apache2 -k start
14069  1079  148.7 MB  3.4 MB   /usr/sbin/apache2 -k start
14089  1079  145.6 MB  0.2 MB   /usr/sbin/apache2 -k start
14127  1079  145.6 MB  0.2 MB   /usr/sbin/apache2 -k start
14132  1079  145.6 MB  0.2 MB   /usr/sbin/apache2 -k start
14149  1079  145.6 MB  0.2 MB   /usr/sbin/apache2 -k start
14481  1079  145.6 MB  0.2 MB   /usr/sbin/apache2 -k start
14548  1079  145.6 MB  0.2 MB   /usr/sbin/apache2 -k start
14552  1079  145.6 MB  0.2 MB   /usr/sbin/apache2 -k start
14553  1079  145.6 MB  0.2 MB   /usr/sbin/apache2 -k start
14554  1079  146.7 MB  3.4 MB   /usr/sbin/apache2 -k start
14555  1079  145.6 MB  0.2 MB   /usr/sbin/apache2 -k start
14556  1079  145.6 MB  0.2 MB   /usr/sbin/apache2 -k start
14557  1079  145.6 MB  0.2 MB   /usr/sbin/apache2 -k start
14593  1079  148.8 MB  3.6 MB   /usr/sbin/apache2 -k start
14594  1079  159.8 MB  14.4 MB  /usr/sbin/apache2 -k start
14606  1079  146.7 MB  3.4 MB   /usr/sbin/apache2 -k start
14792  1079  149.0 MB  3.8 MB   /usr/sbin/apache2 -k start
14808  1079  145.6 MB  0.2 MB   /usr/sbin/apache2 -k start
14914  1079  145.6 MB  0.2 MB   /usr/sbin/apache2 -k start
14915  1079  145.6 MB  0.2 MB   /usr/sbin/apache2 -k start
14917  1079  145.6 MB  0.2 MB   /usr/sbin/apache2 -k start
14918  1079  145.9 MB  0.3 MB   /usr/sbin/apache2 -k start
14920  1079  145.6 MB  0.2 MB   /usr/sbin/apache2 -k start
14927  1079  145.6 MB  0.2 MB   /usr/sbin/apache2 -k start
14997  1079  145.6 MB  0.2 MB   /usr/sbin/apache2 -k start
14998  1079  159.7 MB  14.4 MB  /usr/sbin/apache2 -k start
14999  1079  149.0 MB  3.8 MB   /usr/sbin/apache2 -k start
15016  1079  145.6 MB  0.2 MB   /usr/sbin/apache2 -k start
15239  1079  145.9 MB  0.4 MB   /usr/sbin/apache2 -k start
15244  1079  145.9 MB  0.3 MB   /usr/sbin/apache2 -k start
15249  1079  145.6 MB  0.2 MB   /usr/sbin/apache2 -k start
15250  1079  145.6 MB  0.2 MB   /usr/sbin/apache2 -k start
15251  1079  145.6 MB  0.2 MB   /usr/sbin/apache2 -k start
15253  1079  145.6 MB  0.2 MB   /usr/sbin/apache2 -k start
15254  1079  154.5 MB  9.0 MB   /usr/sbin/apache2 -k start
15255  1079  145.6 MB  0.2 MB   /usr/sbin/apache2 -k start
15256  1079  150.2 MB  4.6 MB   /usr/sbin/apache2 -k start
15398  1079  145.8 MB  1.0 MB   /usr/sbin/apache2 -k start
15405  1079  146.9 MB  3.7 MB   /usr/sbin/apache2 -k start
15406  1079  160.0 MB  14.6 MB  /usr/sbin/apache2 -k start
15407  1079  160.9 MB  15.4 MB  /usr/sbin/apache2 -k start
15442  1079  145.6 MB  0.2 MB   /usr/sbin/apache2 -k start
15786  1079  145.6 MB  0.2 MB   /usr/sbin/apache2 -k start
15906  1079  145.6 MB  0.2 MB   /usr/sbin/apache2 -k start
### Processes: 85
### Total private dirty RSS: 282.41 MB


-------- Nginx processes --------

### Processes: 0
### Total private dirty RSS: 0.00 MB


------ Passenger processes ------
PID   VMSize     Private    Name
---------------------------------
1085  215.8 MB   0.3 MB     PassengerWatchdog
1093  2021.7 MB  4.1 MB     PassengerHelperAgent
1099  109.7 MB   6.7 MB     Passenger spawn server
1102  159.2 MB   0.8 MB     PassengerLoggingAgent
1816  563.2 MB   325.2 MB   Rack: 
/home/guarddog/public_html/guarddog.com/current
2593  541.4 MB   303.4 MB   Rack: 
/home/guarddog/public_html/guarddog.com/current
4794  354.3 MB   118.0 MB   Rack: 
/home/guarddog/public_html/guarddog.com/current
4829  455.9 MB   170.8 MB   Rack: 
/home/guarddog/public_html/guarddog.com/current
4925  1327.5 MB  1040.0 MB  Rack: 
/home/guarddog/public_html/guarddog.com/current
5421  1348.8 MB  1110.6 MB  Rack: 
/home/guarddog/public_html/guarddog.com/current
### Processes: 10
### Total private dirty RSS: 3079.98 MB

Do you notice how those two rack processes 4925 and 5421 are using an 
incredible amount of memory, much more than anything else? Obviously, this 
is the source of the issue. But the question I have is why. It leaves me 
virtually with no memory left on the machine:

$ cat meminfo
MemTotal:        6113156 kB
MemFree:          510920 kB

I looked at my server production log and noticed users were exporting data 
within a specific date range, say a week, which is probably 1 thousand 
records of data. But still that shouldn''t cause so much memory
consumption.
Also emails are sent out quite frequently, but still I worked on other 
rails applications, and I never saw email sending consume that much memory. 
Note I also create a new thread when sending emails:

def deliver_async(email)
  Thread.new do
    email.deliver
  end
end

I am not sure the next step to resolve why those two rack processes are 
consuming so much memory. A lot of the memory leak tools I see our outdated 
for rails 2 or older. THe rails guides mentions valgrind but that is for 
testing isolated lines of code, and I am hoping for a quicker solution. I 
also read about "ObjectSpace.count_objects". Is there an ideal place
to
place this line of code in the application?





-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to
rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit
https://groups.google.com/d/msgid/rubyonrails-talk/f5d6a53d-7f6a-4025-b68c-8fdf7b3e0377%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Jim

2013-Dec-05 14:37 UTC

head link

Re: rails and passenger memory issues

On Tuesday, December 3, 2013 8:03:51 PM UTC-5, John Merlino wrote:
> I''m using Rails 3, Passenger, and Postgresql. Yesterday the rails 
> application was running very slow. So I rebooted the server. Less than 24 
> hours later, it is consuming even more memory than before and when trying 
> to load the web page, it just hangs forever. I ran passenger-memory-stats 
> to inspect the issue. And once again, I notice 2 rack processes using over 
> 2 gigabytes of private dirty RSS:
>
This blog post mentions using the xray gem to inspect running processes. 
 That may or may not help, since you don''t know if they''re
still doing
whatever caused them to use so much memory by the time you find them. 
 Couldn''t hurt to try, though.

http://pivotallabs.com/inspect-running-ruby-processes-using-xray-and-kill-3/

Jim Crate

-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to
rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit
https://groups.google.com/d/msgid/rubyonrails-talk/c6e18c0a-45a2-4d29-88d5-95910f478ae6%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Rails - Dec 2013 - rails and passenger memory issues

rails and passenger memory issues

Re: rails and passenger memory issues