Hi everyone,
I found and tried two optimization hacks; however, I don''t feel too
comfortable with them. Therefore I would appreciate your opinion:
- Caching proc objects, i.e.
class Symbol
def to_proc
@to_proc ||= Proc.new { |*args| args.shift.__send__(self, *args) }
end
end
- and disabling the GC during an action:
around_filter do |controller, action|
GC.disable
begin
action.call
ensure
GC.enable
GC.start
end
end
On some of our machines and with some requests they boost performance
by as much as 50%. however, as I don''t know if I shuld feel ssave with
them I would like to hear your opinions.
BTW: A longer version of this text, with some explanation, is here:
http://1rad.wordpress.com/2008/11/10/0x0a-some-optimization-hacks/
/eno
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---
Some questions about disabling GC: * Memory consuming action - will it kill server? * Very small (not memory consuming) action - won''t GC.start slow it down significantly? (it''s called every time) * Lets assume that one of your libraries used by the action is calling GC.start explicitly - the library "knows" it consumes a lot of memory and tries to free it in the right time - what will happen? These are just some random questions about "bad" scenarios. On Nov 12, 6:02 pm, "Enrico Thierbach" <enrico.thierb...-gM/Ye1E23mwN+BqQ9rBEUg@public.gmane.org> wrote:> Hi everyone, > > I found and tried two optimization hacks; however, I don''t feel too > comfortable with them. Therefore I would appreciate your opinion: > > - Caching proc objects, i.e. > > class Symbol > def to_proc > @to_proc ||= Proc.new { |*args| args.shift.__send__(self, *args) } > end > end > > - and disabling the GC during an action: > > around_filter do |controller, action| > GC.disable > begin > action.call > ensure > GC.enable > GC.start > end > end > > On some of our machines and with some requests they boost performance > by as much as 50%. however, as I don''t know if I shuld feel ssave with > them I would like to hear your opinions. > > BTW: A longer version of this text, with some explanation, is here:http://1rad.wordpress.com/2008/11/10/0x0a-some-optimization-hacks/ > > /eno--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Hi reHa, On Wed, Nov 12, 2008 at 9:49 PM, reHa <preszke-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> > Some questions about disabling GC: > > * Memory consuming action - will it kill server? > > * Lets assume that one of your libraries used by the action is calling > GC.start explicitly - the library "knows" it consumes a lot of memory > and tries to free it in the right time - what will happen?Hell, for sure those are dangerous situations. But then I guess I would know about the memory consumption of my app, or measure it in a high usage scenario. I guess this is an issue for all apps, GC running or not: If you are running some code that needs much RAM all at the same time (so even the GC kicking in couldn''t help) this can already kill your server. (And I have seen ActiveRecord queries taking up up to somewhere close to 1 GByte) So you should always know what memory requirements your app has. And if you are know that and know that you are on the right side of the tracks, then, I guess, you should be fine. In our app we have quite a strict upper limit on how many objects we have to deal with, so I guess I might feel safe :)> * Very small (not memory consuming) action - won''t GC.start slow it > down significantly? (it''s called every time)Yes, certainly. On my developer machine a GC run takes ~200msecs, which would turn fast actions in pretty slow ones :) So I would like to have the GC run somewhere in the rails kernel - *after* the response would be sent back to the client. But that, of course, can only be done by patching rails. And of course I wouldn''t run that code on all actions, only on some anyways.> These are just some random questions about "bad" scenarios.Thanks for your response. Did anyone use techniques like that before? /eno ===================================================================A wee piece of ruby every monday: http://1rad.wordpress.com/ --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
>> Some questions about disabling GC: >> >> * Memory consuming action - will it kill server? >> >> * Lets assume that one of your libraries used by the action is >> calling >> GC.start explicitly - the library "knows" it consumes a lot of memory >> and tries to free it in the right time - what will happen? > > Hell, for sure those are dangerous situations. But then I guess I > would know about the memory consumption of my app, or measure it in a > high usage scenario. > > I guess this is an issue for all apps, GC running or not: If you are > running some code that needs much RAM all at the same time (so even > the GC kicking in couldn''t help) this can already kill your server. > (And I have seen ActiveRecord queries taking up up to somewhere close > to 1 GByte) So you should always know what memory requirements your > app has. And if you are know that and know that you are on the right > side of the tracks, then, I guess, you should be fine. > > In our app we have quite a strict upper limit on how many objects we > have to deal with, so I guess I might feel safe :)Be careful here... I wrote an app awhile back that used the ultraviolet syntax highlighter. Didn''t even occur to me it might chew up a lot of ram and in all my usage it didn''t. Until I sent it a couple pages of horribly formatted html and it ballooned into a couple hundred megs. Kind of surprised me. Point being.. just be sure to double check your assumptions about memory usage... everyone knows to take another look at imagemagick, but some others suffer in odd cases as well... Also, as to the Proc enhancement, I haven''t thought about it and my head hurts... But what happens if one of the arguments is say Time.now? Will that get evaluated *once* or each time? That is, will you have the same issue you''d have if you wrote: named_scope :foo, :conditions => ["created_at >= ?", 5.days.ago] Where 5.days.ago gets evaulated *once* and never changes until you restart (hence the need to passing a lambda into your namedscope). -philip --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Hi philip,> Also, as to the Proc enhancement, I haven''t thought about it and my > head hurts... > > But what happens if one of the arguments is say Time.now? Will that > get evaluated *once* or each time? That is, will you have the same > issue you''d have if you wrote: > > named_scope :foo, :conditions => ["created_at >= ?", 5.days.ago] > > Where 5.days.ago gets evaulated *once* and never changes until you > restart (hence the need to passing a lambda into your namedscope).as far as I can see, my code would never be called in those circumstances, as 5.days doesn''t evaluate to a symbol, does it? Comparing Rails'' original implementation def to_proc Proc.new { |*args| args.shift.__send__(self, *args) } end with the improved(?) one: def to_proc @to_proc ||= Proc.new { |*args| args.shift.__send__(self, *args) } end you have the same values "going" into the Proc, namely the symbol (i.e. self). What I don''t know The question is more if there is something "invisible" going on in there? Speaking of questions: does anyone know in what circumstances the *args array can hold any values at all? regards, /eno --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
On Nov 13, 10:31 am, "Enrico Thierbach" <enrico.thierb...-gM/Ye1E23mwN+BqQ9rBEUg@public.gmane.org> wrote:> def to_proc > Proc.new { |*args| args.shift.__send__(self, *args) } > end > > with the improved(?) one: > > def to_proc > @to_proc ||= Proc.new { |*args| args.shift.__send__(self, *args) } > endThis is interesting. I can''t remember quite what it is that makes .map(&:name) slower than .map{|o| o.name} But I have a feeling that it was to do with the cost of instantiating a new Proc object each time. With your solution, I''d be interested in seeing some benchmarks. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Sure: there you go, in a somewhat :) clinical testcase.
----------------- snip -----------------
require "benchmark"
CNT=200000
Benchmark.bm { |x|
x.report "V1" do
(1..CNT).each do [1].each { |i| i.to_i } end
end
}
class Symbol
def to_proc; Proc.new { |*args| args.shift.__send__(self, *args) }; end
end
Benchmark.bm { |x|
x.report "V2" do
(1..CNT).each do [].each (&:to_i) end
end
}
class Symbol
def to_proc; Proc.new { |obj| obj.__send__(self) }; end
end
Benchmark.bm { |x|
x.report "V3" do
(1..CNT).each do [1].each (&:to_i) end
end
}
class Symbol
def to_proc; @proc ||= Proc.new { |*args| args.shift.__send__(self,
*args) }; end
end
Benchmark.bm { |x|
x.report "V4" do
(1..CNT).each do [1].each (&:to_i) end
end
}
----------------- snap ----------------
The result is on my machine as follows:
user system total real
V1 0.200000 0.000000 0.200000 ( 0.201815)
user system total real
V2 0.860000 0.000000 0.860000 ( 0.870993)
user system total real
V3 0.950000 0.000000 0.950000 ( 0.953523)
user system total real
V4 0.680000 0.010000 0.690000 ( 0.685361)
The fastest, V1, doesn''t build proc objects at all. V2 is
rails''
default implementation, V4 caches the Proc objects inside the symbol,
i.e. generates the Proc object only.
Of course, invoking 200000 Proc objects this way is usually not what
you are doing in any application; we, however, had a test case where
we did it 9000 times per action and it matched badly with the garbage
collector. Ruby enterprise behaved much much better :)
/eno
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---