Rails - May 2006 - Possible solutions to txtdrive process killing for cpu usage

Hi,

I''ve just deployed a basic ecommerce site on textdrive.com. The 
application has been stress tested on our local development machines and 
appears to be running fine (no memory leaks).

However, on textdrive the fcgi processes are being killed after just a 
few requests apparently because of cpu usage (The log says the processes 
exceeds 17% average cpu usage in the last few minutes)

At first it seemed that they we''re being killed because the processes 
were exceediing the 48Mb resident memory size limit (RSS seemed to 
suddenly increase on certain pages (mostly backoffice functions) though 
not consistently) but I was pointed to a process_watchdog.log filr which 
points to the cpu usage.

Has anyone come across a similar situation? Any hints suggestions on 
what I can do about it?

I''d really appreciate any help as although I''m now doing a
optimization
refactoring iteration I''m uncertain how I can avoid this.

Regards,

Saimon Moore

-- 
Posted via http://www.ruby-forum.com/.

I think you should post this to the textdrive forum (http:// 
forum.textdrive.com/).

On May 18, 2006, at 7:54 PM, Saimon Moore wrote:
> Hi,
>
> I''ve just deployed a basic ecommerce site on textdrive.com. The
> application has been stress tested on our local development  
> machines and
> appears to be running fine (no memory leaks).
>
> However, on textdrive the fcgi processes are being killed after just a
> few requests apparently because of cpu usage (The log says the  
> processes
> exceeds 17% average cpu usage in the last few minutes)
>
> At first it seemed that they we''re being killed because the
processes
> were exceediing the 48Mb resident memory size limit (RSS seemed to
> suddenly increase on certain pages (mostly backoffice functions)  
> though
> not consistently) but I was pointed to a process_watchdog.log filr  
> which
> points to the cpu usage.
>
> Has anyone come across a similar situation? Any hints suggestions on
> what I can do about it?
>
> I''d really appreciate any help as although I''m now doing
a
> optimization
> refactoring iteration I''m uncertain how I can avoid this.
>
> Regards,
>
> Saimon Moore
>
> -- 
> Posted via http://www.ruby-forum.com/.
> _______________________________________________
> Rails mailing list
> Rails@lists.rubyonrails.org
> http://lists.rubyonrails.org/mailman/listinfo/rails
Dae San Hwang
daesan@gmail.com

I already have but what I''m asking for on this list is help or 
suggestions with how I could go about tracking/reducing the cpu usage 
problem which is rails related.

Dae San Hwang wrote:
> I think you should post this to the textdrive forum (http://
> forum.textdrive.com/).
> 
> On May 18, 2006, at 7:54 PM, Saimon Moore wrote:
> 
>> exceeds 17% average cpu usage in the last few minutes)
>> what I can do about it?
>> Posted via http://www.ruby-forum.com/.
>> _______________________________________________
>> Rails mailing list
>> Rails@lists.rubyonrails.org
>> http://lists.rubyonrails.org/mailman/listinfo/rails
> 
> Dae San Hwang
> daesan@gmail.com

-- 
Posted via http://www.ruby-forum.com/.

On May 18, 2006, at 4:17 AM, Saimon Moore wrote:
> I already have but what I''m asking for on this list is help or
> suggestions with how I could go about tracking/reducing the cpu usage
> problem which is rails related.
>
> Dae San Hwang wrote:
>> I think you should post this to the textdrive forum (http://
>> forum.textdrive.com/).
>>
>> On May 18, 2006, at 7:54 PM, Saimon Moore wrote:
>>
>>> exceeds 17% average cpu usage in the last few minutes)
>>> what I can do about it?
>>> Posted via http://www.ruby-forum.com/.
>>> _______________________________________________
>>> Rails mailing list
>>> Rails@lists.rubyonrails.org
>>> http://lists.rubyonrails.org/mailman/listinfo/rails
>>
>> Dae San Hwang
>> daesan@gmail.com
>
	Well, what does your app do? What parts of it are processor  
intensive? Using RMagick? Some details and we might be able to help you.

-Ezra

Hi Ezra,

This is what I find strange. Just to choose one example, I have an edit 
product view whose controller action simply finds the product and makes 
an instance variable out of it so that the form can be populated (i.e. 
it''s not the product update action).

e.g.

    @product = Product.find(params[:id])
    @song = Song.new
    unless @product && @product.sellable
      @artists = String.new
      @label = Label.new
      @genre = Genre.new
      @instrument = Instrument.new
      @supplier = Supplier.new
    else
      @artists = @product.sellable.artists.collect {|a| a.name }.join(''
, '')
      @label = @product.sellable.label.name
      @genre = @product.sellable.genre.name
      @instrument = @product.sellable.instrument.name
      @supplier = @product.supplier.name
    end

The view is just a basic form (created from a few partials) utilizing 
standard rails form tags to edit the product. It has a small section 
specific to the kind of sellable and a section specific to product 
details.

I have suspected Rmagick as a possible offender but after further tests 
I have rejected it as a possibility. (Images are processed via rmagick 
just the once and thereafter they are cached on the browser)

The Product model:

  has_one :sellable
  belongs_to :supplier #where supplier is sublclassed into Album/Song 
(STI)
  belongs_to :image
  has_many :shopping_items, :dependent => :destroy
  has_many :supplier_order_items
  belongs_to :user

There''s nothing very complex about this action apart from the fact that
the AR model has a few associations.

I have stress tested this action on my development box (set to 
production mode) and it appears rock solid. The fcgi process has a 
steady 28Mb during and after a heavy attack (ab -kc 50 -t 300). cpu 
levels are similarly non-interesting. valgrind memcheck  doesn''t report
anything out of the ordinary

On textdrive though it''s a different story, rss starts of at 32Mb and 
rockets up after only a few refreshes of this action (and yes I''m 
running the app in production mode), the cpu cycles also increase, 
though up until I say the process_watchdog.log I wasn''t that worried 
about them.

In short I''m stumped as to why this is behaving so differently on 
textdrive and I''m not really sure what I can do to optimize this.

I''d appreciate any advice.

Thanks,

Saimon
Ezra Zygmuntowicz wrote:
> On May 18, 2006, at 4:17 AM, Saimon Moore wrote:
> 
>>>> exceeds 17% average cpu usage in the last few minutes)
>>>> what I can do about it?
>>>> Posted via http://www.ruby-forum.com/.
>>>> _______________________________________________
>>>> Rails mailing list
>>>> Rails@lists.rubyonrails.org
>>>> http://lists.rubyonrails.org/mailman/listinfo/rails
>>>
>>> Dae San Hwang
>>> daesan@gmail.com
>>
> 
> 	Well, what does your app do? What parts of it are processor
> intensive? Using RMagick? Some details and we might be able to help you.
> 
> -Ezra

-- 
Posted via http://www.ruby-forum.com/.

Hi !

On May 18, 2006, at 9:55 AM, Saimon Moore wrote:
> Hi Ezra,
>
> This is what I find strange. Just to choose one example, I have an  
> edit
> product view whose controller action simply finds the product and  
> makes
> an instance variable out of it so that the form can be populated (i.e.
> it''s not the product update action).
>
> e.g.
>
>     @product = Product.find(params[:id])
>     @song = Song.new
>     unless @product && @product.sellable
>       @artists = String.new
>       @label = Label.new
>       @genre = Genre.new
>       @instrument = Instrument.new
>       @supplier = Supplier.new
>     else
>       @artists = @product.sellable.artists.collect {|a|  
> a.name }.join(''
> , '')
>       @label = @product.sellable.label.name
>       @genre = @product.sellable.genre.name
>       @instrument = @product.sellable.instrument.name
>       @supplier = @product.supplier.name
>     end
>
> The view is just a basic form (created from a few partials) utilizing
> standard rails form tags to edit the product. It has a small section
> specific to the kind of sellable and a section specific to product
> details.
>
> I have suspected Rmagick as a possible offender but after further  
> tests
> I have rejected it as a possibility. (Images are processed via rmagick
> just the once and thereafter they are cached on the browser)
>
> The Product model:
>
>   has_one :sellable
>   belongs_to :supplier #where supplier is sublclassed into Album/Song
> (STI)
>   belongs_to :image
>   has_many :shopping_items, :dependent => :destroy
>   has_many :supplier_order_items
>   belongs_to :user
>
> There''s nothing very complex about this action apart from the fact
> that
> the AR model has a few associations.
>
> I have stress tested this action on my development box (set to
> production mode) and it appears rock solid. The fcgi process has a
> steady 28Mb during and after a heavy attack (ab -kc 50 -t 300). cpu
> levels are similarly non-interesting. valgrind memcheck  doesn''t  
> report
> anything out of the ordinary
>
> On textdrive though it''s a different story, rss starts of at 32Mb
and
> rockets up after only a few refreshes of this action (and yes I''m
> running the app in production mode), the cpu cycles also increase,
> though up until I say the process_watchdog.log I wasn''t that
worried
> about them.
>
> In short I''m stumped as to why this is behaving so differently on
> textdrive and I''m not really sure what I can do to optimize this.
>
> I''d appreciate any advice.
>
> Thanks,
>
> Saimon


Saimon-

	Yeah that sounds a bit fishy when it works fine on your local box  
but not on txtdrv server. What OS are you running locally? I don''t  
think I have any clear cut answers for you right off the bat here.  
Since it seems like a textdrive problem you should probably file a  
support ticket there detailing how it works locally but doesn''t work  
on their servers.

	I no longer use textdrive so I can''t say what the issue might be.  
Hope you get it figured out though.

Cheers-
-Ezra

Yes it''s very strange which is why I''m stumped.

I''m running Gentoo Linux 2.6.16r4.

I have opened the ticket with them but atm they just point me to the 
process_watchdog.log.

You mentioned you''re no longer with textdrive. Could I ask who
you''re
with?

Thanks again for helping out...

Regards,

Saimon


Ezra Zygmuntowicz wrote:
> Hi !
> 
> On May 18, 2006, at 9:55 AM, Saimon Moore wrote:
> 
>>
>> a.name }.join(''
>> details.
>> (STI)
>> production mode) and it appears rock solid. The fcgi process has a
>>
>> In short I''m stumped as to why this is behaving so differently
on
>> textdrive and I''m not really sure what I can do to optimize
this.
>>
>> I''d appreciate any advice.
>>
>> Thanks,
>>
>> Saimon
> 
> 
> 
> Saimon-
> 
> 	Yeah that sounds a bit fishy when it works fine on your local box
> but not on txtdrv server. What OS are you running locally? I don''t
> think I have any clear cut answers for you right off the bat here.
> Since it seems like a textdrive problem you should probably file a
> support ticket there detailing how it works locally but doesn''t
work
> on their servers.
> 
> 	I no longer use textdrive so I can''t say what the issue might be.
> Hope you get it figured out though.
> 
> Cheers-
> -Ezra

-- 
Posted via http://www.ruby-forum.com/.

On May 18, 2006, at 11:08 AM, Saimon Moore wrote:
> Yes it''s very strange which is why I''m stumped.
>
> I''m running Gentoo Linux 2.6.16r4.
>
> I have opened the ticket with them but atm they just point me to the
> process_watchdog.log.
>
> You mentioned you''re no longer with textdrive. Could I ask who
you''re
> with?
>
> Thanks again for helping out...
>
> Regards,
>
> Saimon
>
Saimon-

	I host a bunch of rails stuff on Xen based VPS''s with http:// 
rimuhosting.com  It is so much nicer to have your own separate  
environment to run your rails apps in without anyone else killing  
your processes and such.

	I have written a tutorial a while ago about setting up a debian vps  
from scratch here:  http://brainspl.at/rails_stack.html

	If you don''t want to be your own sysadmin I also like PlanetArgon  
for shared hosting.

-Ezra

I''ve had a look at rimuhosting.com. Seems quite interesting. As this 
application is for a client of mine and at the moment I''m using my own 
personal sharing plan to host it, I''m probably going to recommend a 
hosting change if things don''t pan out with textdrive (I was going to
do
that sooner or later) so I''ll be evaluating it and Planet Argon too.

Nice tutorial...

Thanks for the info...

Ezra Zygmuntowicz wrote:
> On May 18, 2006, at 11:08 AM, Saimon Moore wrote:
> 
>> Thanks again for helping out...
>>
>> Regards,
>>
>> Saimon
>>
> 
> Saimon-
> 
> 	I host a bunch of rails stuff on Xen based VPS''s with http://
> rimuhosting.com  It is so much nicer to have your own separate
> environment to run your rails apps in without anyone else killing
> your processes and such.
> 
> 	I have written a tutorial a while ago about setting up a debian vps
> from scratch here:  http://brainspl.at/rails_stack.html
> 
> 	If you don''t want to be your own sysadmin I also like PlanetArgon
> for shared hosting.
> 
> -Ezra

-- 
Posted via http://www.ruby-forum.com/.

On May 18, 2006, at 9:55 AM, Saimon Moore wrote:
> I have suspected Rmagick as a possible offender but after further  
> tests
> I have rejected it as a possibility. (Images are processed via rmagick
> just the once and thereafter they are cached on the browser)
FYI, historically Rmagick has been the source of many issues on  
TextDrive because usage can go through the roof. Also, many TxD  
servers are 64-bit, which would have different memory usage patterns  
than you may expect.

-- 
_Deirdre                                             http://deirdre.net