So here''s a random question: if a (Ruby) multi-threaded rails server could exist (bug free), would it be faster than using a mongrel cluster? (I.e. 10 mongrel processes versus 10 Ruby threads). I''m not sure if it would. RAM it might save, though. Any thoughts? -Roger -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20070907/d2fed9f0/attachment.html
On Sep 7, 2007, at 11:33 AM, Roger Pack wrote:> So here''s a random question: if a (Ruby) multi-threaded rails > server could exist (bug free), would it be faster than using a > mongrel cluster? (I.e. 10 mongrel processes versus 10 Ruby > threads). I''m not sure if it would. RAM it might save, though. Any > thoughts? -Roger > _______________________________________________ > Mongrel-users mailing list > Mongrel-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-usersHey Roger- No it would not be as fast at all. Current ruby threads are green threads, meaning that they do not use native OS threads so there is no real parallel execution. Ruby has an internal timer and just switches between threads really fast. So 10 mongrels will trounce one thread safe mongrel. Ruby 1.9, Jruby and Rubinius will eventually have native threads and may make this situation better but for now such is life. Cheers- -- Ezra Zygmuntowicz -- Founder & Ruby Hacker -- ez at engineyard.com -- Engine Yard, Serious Rails Hosting -- (866) 518-YARD (9273)
At Fri, 7 Sep 2007 11:43:24 -0700, Ezra Zygmuntowicz <ezmobius at gmail.com> wrote:> Hey Roger- > > No it would not be as fast at all. Current ruby threads are green > threads, meaning that they do not use native OS threads so there is > no real parallel execution. Ruby has an internal timer and just > switches between threads really fast. So 10 mongrels will trounce one > thread safe mongrel. > > Ruby 1.9, Jruby and Rubinius will eventually have native > threads and may make this situation better but for now such is life.I don?t want to muddy the waters here, as my knowledge of Ruby threading performance is minimal, but there isn?t any ?real? parallel execution on any uniprocessor machine: it?s all emulation. Erlang uses green threads (but calls them processes) but context switches quite a bit faster (if I understand correctly) than native threads or processes. True, on a multiprocessor machine Ruby isn?t going to take advantage of the situation without native threads or running multiple processes. But it may turn out that a mixture of multiple processes AND green threads provides greater performance than plain old multiple processes. This paper, for instance, from 2002, claims significantly better performance on the JVM (Linux) with green threads than native. <http://cselab.snu.ac.kr/member/naehyuck/papers/%5BJ14%5DComparative%20performance%20evaluation%20of%20Java%20threads%20for%20embedded%20applications.pdf> best, Erik Hetzner ;; Erik Hetzner, California Digital Library ;; gnupg key id: 1024D/01DB07E3 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://rubyforge.org/pipermail/mongrel-users/attachments/20070907/129a6b26/attachment.bin
Here''s one question that I think would have an impact on how useful green threads are for many server apps: If you are blocking on something like a database SELECT in one green thread, will the Ruby interpreter switch to another green thread while it''s waiting for the response? Or does a blocking request block all green threads? Thanks, Pete On Sep 7, 2007, at 12:02 PM, Erik Hetzner wrote:> At Fri, 7 Sep 2007 11:43:24 -0700, > Ezra Zygmuntowicz <ezmobius at gmail.com> wrote: >> Hey Roger- >> >> No it would not be as fast at all. Current ruby threads are green >> threads, meaning that they do not use native OS threads so there is >> no real parallel execution. Ruby has an internal timer and just >> switches between threads really fast. So 10 mongrels will trounce one >> thread safe mongrel. >> >> Ruby 1.9, Jruby and Rubinius will eventually have native >> threads and may make this situation better but for now such is life. > > I don?t want to muddy the waters here, as my knowledge of Ruby > threading performance is minimal, but there isn?t any ?real? parallel > execution on any uniprocessor machine: it?s all emulation. Erlang uses > green threads (but calls them processes) but context switches quite a > bit faster (if I understand correctly) than native threads or > processes. True, on a multiprocessor machine Ruby isn?t going to take > advantage of the situation without native threads or running multiple > processes. But it may turn out that a mixture of multiple processes > AND green threads provides greater performance than plain old multiple > processes. > > This paper, for instance, from 2002, claims significantly better > performance on the JVM (Linux) with green threads than native. > <http://cselab.snu.ac.kr/member/naehyuck/papers/%5BJ14%5DComparative > %20performance%20evaluation%20of%20Java%20threads%20for%20embedded% > 20applications.pdf> > > best, > Erik Hetzner > ;; Erik Hetzner, California Digital Library > ;; gnupg key id: 1024D/01DB07E3 > _______________________________________________ > Mongrel-users mailing list > Mongrel-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-users
On 9/7/07, Erik Hetzner <erik.hetzner at ucop.edu> wrote:> processes. True, on a multiprocessor machine Ruby isn''t going to take > advantage of the situation without native threads or running multiple > processes. But it may turn out that a mixture of multiple processes > AND green threads provides greater performance than plain old multiple > processes.I can state with a great deal of confidence that in this problem domain, Ruby''s green threading will not get you greater throughput. A given process can only do so much work. If, in the course of doing that work, there is an external latency, such as waiting on some IO, that can be done in a non-blocking way, then it''s possible to get more work out of that process per unit time by using threads. The work involved in handling web browser requests and rendering responses back to them, though, doesn''t tend to fall into this category. It tends to be CPU intensive, and when one is waiting for something external, like a database query, one is usually waiting inside the body of the low level extension based database driver, so thread context switching doesn''t happen there, anyway. The most efficient way of dealing with these sorts of things, from a throughput perspective, is to avoid the overhead of threads completely and just pound them through, one at a time, with 100% of the process resources working on that single request at a time. This doesn''t change if you have one process or ten processes for your app/site. Kirk Haines swiftiply.swiftcore.org analogger.swiftcore.org
On 9/7/07, Pete DeLaurentis <pete at nextengine.com> wrote:> Here''s one question that I think would have an impact on how useful > green threads are for many server apps: > > If you are blocking on something like a database SELECT in one green > thread, will the Ruby interpreter switch to another green thread > while it''s waiting for the response? > > Or does a blocking request block all green threads?When the flow of execution leaves Ruby and goes into some 3rd party extension (like that of a DB library), then ruby can''t context switch. So yes, when one does a DB query that is going through a C extension, all of the other green threads are blocked, so that latency can''t be captured. If it could be, then yes, there would be some benefit from some number of green threads per process for typical web apps. Kirk Haines
Are you assuming that the memory hit per-thread is about the same as per mongrel process? -faisal On Sep 7, 2007, at 2:43 PM, Ezra Zygmuntowicz wrote:> > On Sep 7, 2007, at 11:33 AM, Roger Pack wrote: > >> So here''s a random question: if a (Ruby) multi-threaded rails >> server could exist (bug free), would it be faster than using a >> mongrel cluster? (I.e. 10 mongrel processes versus 10 Ruby >> threads). I''m not sure if it would. RAM it might save, though. Any >> thoughts? -Roger >> _______________________________________________ >> Mongrel-users mailing list >> Mongrel-users at rubyforge.org >> http://rubyforge.org/mailman/listinfo/mongrel-users > > > Hey Roger- > > No it would not be as fast at all. Current ruby threads are green > threads, meaning that they do not use native OS threads so there is > no real parallel execution. Ruby has an internal timer and just > switches between threads really fast. So 10 mongrels will trounce one > thread safe mongrel. > > Ruby 1.9, Jruby and Rubinius will eventually have native threads and > may make this situation better but for now such is life. > > Cheers- > -- Ezra Zygmuntowicz > -- Founder & Ruby Hacker > -- ez at engineyard.com > -- Engine Yard, Serious Rails Hosting > -- (866) 518-YARD (9273) > > > _______________________________________________ > Mongrel-users mailing list > Mongrel-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-users
On 9/7/07, Faisal N Jawdat <faisal at faisal.com> wrote:> Are you assuming that the memory hit per-thread is about the same as > per mongrel process?No. It''s irrelevant, because there''s only so much work that a single ruby process can do, regardless of the number of threads it contains. Once the process is doing all of the work that it can, you only get more work done by adding another process. Repeat until all of your CPUs are doing all of the work that they can. I have a personal hunch that most of the time people are running more Mongrels than they need to run to maximize the throughput on their hardware, though. Anyone who is concerned about the RAM footprint of their application should step back and take a look at how many mongrel processes they are running. Do some analysis and see if they can reduce that number without affecting throughput. Maybe take a look at using Swiftiply+swiftiplied_mongrel evented_mongrel instead of the threaded mongrel and look again. I think that in many cases a person can run a lot fewer processes than they are, and still get the same, or maybe even better throughput. Once your CPUs are at max utilization, more processes don''t help. Kirk Haines
At Fri, 7 Sep 2007 12:32:22 -0700, "Kirk Haines" <wyhaines at gmail.com> wrote:> > On 9/7/07, Erik Hetzner <erik.hetzner at ucop.edu> wrote: > > processes. True, on a multiprocessor machine Ruby isn''t going to take > > advantage of the situation without native threads or running multiple > > processes. But it may turn out that a mixture of multiple processes > > AND green threads provides greater performance than plain old multiple > > processes. > > I can state with a great deal of confidence that in this problem > domain, Ruby''s green threading will not get you greater throughput.What problem domain? Web servers in general? Web applications? Web applications which are heavily database dependent? I show below that there are web apps whose throughput can be improved by green threading.> A given process can only do so much work. If, in the course of doing > that work, there is an external latency, such as waiting on some IO, > that can be done in a non-blocking way, then it''s possible to get more > work out of that process per unit time by using threads.A process can use almost all the cycles available on a machine if the scheduler allows it to, and if it has something to do. Threads and a thread scheduler allow a process to have something to do.> The work involved in handling web browser requests and rendering > responses back to them, though, doesn''t tend to fall into this > category. It tends to be CPU intensive, and when one is waiting for > something external, like a database query, one is usually waiting > inside the body of the low level extension based database driver, so > thread context switching doesn''t happen there, anyway.In my experience web applications tend to spend most of their time waiting for other things, from databases, web services, etc. So obviously your mileage may vary.> The most efficient way of dealing with these sorts of things, from a > throughput perspective, is to avoid the overhead of threads completely > and just pound them through, one at a time, with 100% of the process > resources working on that single request at a time. > > This doesn''t change if you have one process or ten processes for > your app/site.Below is an example of what I?m getting at. You can tweak it to test various things. It is a simple mongrel server that sleeps for 5 usecs and then calculates the fibonacci number of 15. You can turn it into a multi-threaded server by commented out the synchronize stuff. On my uniprocessor machine you get half again the performance in terms of requests/sec from multithreaded. This probably does not model a typical web app. But my original response was to a blanket statement that it was impossible to get better performance from more threads. Here is an example of a web app that gets better performance with more threads. best, Erik Hetzner ============== require ''rubygems'' require ''mongrel'' require ''monitor'' port = ARGV[0] class MyHttpHandler < Mongrel::HttpHandler @@lock = Monitor.new def fib(n) if n == 1 or n == 2 then return 1 else return fib(n - 1) + fib(n - 2) end end def process(request,response) @@lock.synchronize do sleep 0.005 response.start(200) do |head,body| body << fib(15).to_s end end end end config = Mongrel::Configurator.new :host => "localhost" do listener :port => port do uri "/", :handler => MyHttpHandler.new end end config.run begin while true do sleep 1000000 end rescue Interrupt end ===============;; Erik Hetzner, California Digital Library ;; gnupg key id: 1024D/01DB07E3 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://rubyforge.org/pipermail/mongrel-users/attachments/20070907/d7d2cc73/attachment-0001.bin
On 9/7/07, Erik Hetzner <erik.hetzner at ucop.edu> wrote> What problem domain? Web servers in general? Web applications? Web > applications which are heavily database dependent? I show below > that there are web apps whose throughput can be improved by green > threading.The general area of Ruby web applications and the serving thereof.> A process can use almost all the cycles available on a machine if the > scheduler allows it to, and if it has something to do. Threads and a > thread scheduler allow a process to have something to do.Sure, but the _only_ way to get this with Ruby green threads is if you have threads that are blocking on something within Ruby code, so that Ruby can context switch to a different thread and get something done with it while the first one sits around, waiting. You artificially construct exactly that scenario in your example below. I''ll comment on it more in a minute. In real apps, though, most of the time, that just isn''t happening.> > The work involved in handling web browser requests and rendering > > responses back to them, though, doesn''t tend to fall into this > > category. It tends to be CPU intensive, and when one is waiting for > > something external, like a database query, one is usually waiting > > inside the body of the low level extension based database driver, so > > thread context switching doesn''t happen there, anyway. > > In my experience web applications tend to spend most of their time > waiting for other things, from databases, web services, etc. So > obviously your mileage may vary.And when you are waiting on something like a database, you are doing it within the DB driver extension, so there are no context switches happening -- your green threads are blocked.> Below is an example of what I''m getting at. You can tweak it to test > various things. It is a simple mongrel server that sleeps for 5 usecs > and then calculates the fibonacci number of 15. You can turn it into a > multi-threaded server by commented out the synchronize stuff. On my > uniprocessor machine you get half again the performance in terms of > requests/sec from multithreaded.The reason you get more performance when multithreading is because of that sleep call. It is all happening inside of Ruby, so when running in a multithreaded context, when one thread goes to sleep, the others just get the execution cycles. It''s artificial, though. Read Ruby web apps don''t tend to have latencies like that. When they are waiting, they are waiting inside of things that block the entire Ruby process, and not just waiting for something that is blocking a single thread. Kirk Haines
At Fri, 7 Sep 2007 13:50:50 -0700, "Kirk Haines" <wyhaines at gmail.com> wrote:> > On 9/7/07, Erik Hetzner <erik.hetzner at ucop.edu> wrote > > > What problem domain? Web servers in general? Web applications? Web > > applications which are heavily database dependent? I show below > > that there are web apps whose throughput can be improved by green > > threading. > > The general area of Ruby web applications and the serving thereof.Which in my experience involves waiting for external web services, among other things. If you want to say that this is something that isn?t a typical rails app, fine. I?m telling you that my threads are often i/o bound.> Sure, but the _only_ way to get this with Ruby green threads is if > you have threads that are blocking on something within Ruby code, so > that Ruby can context switch to a different thread and get something > done with it while the first one sits around, waiting. You > artificially construct exactly that scenario in your example below. > I''ll comment on it more in a minute. In real apps, though, most of > the time, that just isn''t happening.I understand how threads work, and I don?t think that my construction is artificial.> And when you are waiting on something like a database, you are doing > it within the DB driver extension, so there are no context switches > happening -- your green threads are blocked.Surely it is possible to write a C extension for Ruby that allows context switching from within it? This is outside my understanding of Ruby threads.> The reason you get more performance when multithreading is because > of that sleep call. It is all happening inside of Ruby, so when > running in a multithreaded context, when one thread goes to sleep, > the others just get the execution cycles. It''s artificial, though. > Read Ruby web apps don''t tend to have latencies like that. When they > are waiting, they are waiting inside of things that block the entire > Ruby process, and not just waiting for something that is blocking a > single thread.I know why we are getting greater performance, and again, I don?t think it?s artificial. But, in any case, it was a demonstration, mostly for the original responder, who claimed that you would never get more performance from Ruby threads. Again, your mileage may vary. And it?s all moot for rails users since I don?t see rails going multithreaded. best, Erik Hetzner ;; Erik Hetzner, California Digital Library ;; gnupg key id: 1024D/01DB07E3 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://rubyforge.org/pipermail/mongrel-users/attachments/20070907/ec5d9f95/attachment.bin
On 9/7/07, Pete DeLaurentis <pete at nextengine.com> wrote:> Here''s one question that I think would have an impact on how useful > green threads are for many server apps: > > If you are blocking on something like a database SELECT in one green > thread, will the Ruby interpreter switch to another green thread > while it''s waiting for the response? > > Or does a blocking request block all green threads?Well considering that a single-threaded process will map to a single kernel thread in Solaris 9+ and Linux 2.6+. A process with green threads that makes a system call will be blocked, until the call returns, that means that all green threads in that process are blocked. (Green threads are now known as fibers) ;) This is one reason why IO intensive workloads, which make intensive use of system calls, should benefit from "real" kernel threads, while CPU intensive workloads can have the edge when implemented using green threads, vs kernel threads. (The context switching cost being lower.)> Thanks, > Pete > > > On Sep 7, 2007, at 12:02 PM, Erik Hetzner wrote: > > > At Fri, 7 Sep 2007 11:43:24 -0700, > > Ezra Zygmuntowicz <ezmobius at gmail.com> wrote: > >> Hey Roger- > >> > >> No it would not be as fast at all. Current ruby threads are green > >> threads, meaning that they do not use native OS threads so there is > >> no real parallel execution. Ruby has an internal timer and just > >> switches between threads really fast. So 10 mongrels will trounce one > >> thread safe mongrel. > >> > >> Ruby 1.9, Jruby and Rubinius will eventually have native > >> threads and may make this situation better but for now such is life. > > > > I don''t want to muddy the waters here, as my knowledge of Ruby > > threading performance is minimal, but there isn''t any ''real'' parallel > > execution on any uniprocessor machine: it''s all emulation. Erlang uses > > green threads (but calls them processes) but context switches quite a > > bit faster (if I understand correctly) than native threads or > > processes. True, on a multiprocessor machine Ruby isn''t going to take > > advantage of the situation without native threads or running multiple > > processes. But it may turn out that a mixture of multiple processes > > AND green threads provides greater performance than plain old multiple > > processes. > > > > This paper, for instance, from 2002, claims significantly better > > performance on the JVM (Linux) with green threads than native. > > <http://cselab.snu.ac.kr/member/naehyuck/papers/%5BJ14%5DComparative > > %20performance%20evaluation%20of%20Java%20threads%20for%20embedded% > > 20applications.pdf> > > > > best, > > Erik Hetzner > > ;; Erik Hetzner, California Digital Library > > ;; gnupg key id: 1024D/01DB07E3 > > _______________________________________________ > > Mongrel-users mailing list > > Mongrel-users at rubyforge.org > > http://rubyforge.org/mailman/listinfo/mongrel-users > > _______________________________________________ > Mongrel-users mailing list > Mongrel-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-users >-- - Brian Gupta http://opensolaris.org/os/project/nycosug/
On 9/7/07, Erik Hetzner <erik.hetzner at ucop.edu> wrote:> > And when you are waiting on something like a database, you are doing > > it within the DB driver extension, so there are no context switches > > happening -- your green threads are blocked. > > Surely it is possible to write a C extension for Ruby that allows > context switching from within it? This is outside my understanding of > Ruby threads.Maybe, but it''s irrelevant. The MySQL/Pg/whatever drivers aren''t written to be aware of Ruby internals.> > The reason you get more performance when multithreading is because > > of that sleep call. It is all happening inside of Ruby, so when > > running in a multithreaded context, when one thread goes to sleep, > > the others just get the execution cycles. It''s artificial, though. > > Read Ruby web apps don''t tend to have latencies like that. When they > > are waiting, they are waiting inside of things that block the entire > > Ruby process, and not just waiting for something that is blocking a > > single thread. > > I know why we are getting greater performance, and again, I don''t > think it''s artificial. But, in any case, it was a demonstration, > mostly for the original responder, who claimed that you would never > get more performance from Ruby threads.Ezra? His response simplified it. Of course there are situations where green threads improve performance. They can be quantified very simply, as I already have. When there are external latencies that are not blocking the entire process, green threads can capture those latencies and turn them into additional work. A sleep() call is such a latency. Any pure ruby code that stays in rubyland and doesn''t go into an external extension is such a latency. A DB query that spends most of its time inside of some C library that is external to Ruby is not such a latency.> Again, your mileage may vary. And it''s all moot for rails users since > I don''t see rails going multithreaded.It''s not very difficult, actually, to do it. Just make a copy of the rails.rb mongrel handler for Rails. Remove the bit that wraps the call to Dispatcher.dispatch() in a @guard.synchronize block. Whether all of Rails is actually threadsafe, and whether one''s code is....that''s another thing, but it''s easy to run Rails in multiple threads under Mongrel if one wants to play with that. Kirk Haines
Hi~ On Sep 7, 2007, at 3:34 PM, Kirk Haines wrote:> On 9/7/07, Erik Hetzner <erik.hetzner at ucop.edu> wrote: > >>> And when you are waiting on something like a database, you are doing >>> it within the DB driver extension, so there are no context switches >>> happening -- your green threads are blocked. >> >> Surely it is possible to write a C extension for Ruby that allows >> context switching from within it? This is outside my understanding of >> Ruby threads. > > Maybe, but it''s irrelevant. The MySQL/Pg/whatever drivers aren''t > written to be aware of Ruby internals. > >>> The reason you get more performance when multithreading is because >>> of that sleep call. It is all happening inside of Ruby, so when >>> running in a multithreaded context, when one thread goes to sleep, >>> the others just get the execution cycles. It''s artificial, though. >>> Read Ruby web apps don''t tend to have latencies like that. When they >>> are waiting, they are waiting inside of things that block the entire >>> Ruby process, and not just waiting for something that is blocking a >>> single thread. >> >> I know why we are getting greater performance, and again, I don''t >> think it''s artificial. But, in any case, it was a demonstration, >> mostly for the original responder, who claimed that you would never >> get more performance from Ruby threads.Yeah I actually said that you would never get more performance out of one mongrel running 10 threads then you would with 10 mongrels running one thread. Just to clarify ;) -- Ezra
At Fri, 7 Sep 2007 15:39:33 -0700, Ezra Zygmuntowicz <ezmobius at gmail.com> wrote:> [?] > > > On 9/7/07, Erik Hetzner <erik.hetzner at ucop.edu> wrote:> [?]> >> I know why we are getting greater performance, and again, I don''t > >> think it''s artificial. But, in any case, it was a demonstration, > >> mostly for the original responder, who claimed that you would never > >> get more performance from Ruby threads. > > Yeah I actually said that you would never get more performance out of > one mongrel running 10 threads then you would with 10 mongrels > running one thread. Just to clarify ;)My apologies for misquoting you, and also apologies for referring to you as ?the original responder? - a bit rude, really. :) Here is your original message:> Hey Roger-> No it would not be as fast at all. Current ruby threads are green > threads, meaning that they do not use native OS threads so there is > no real parallel execution. Ruby has an internal timer and just > switches between threads really fast. So 10 mongrels will trounce > one thread safe mongrel.> Ruby 1.9, Jruby and Rubinius will eventually have native threads and > may make this situation better but for now such is life.I?ve just tested this on my uniprocessor machine. 10 single-threaded mongrels of my sample sleep + fibonacci web server, balanced behind an apache load balancer, versus 1 multithreaded mongrels without a load balancer. The performance, in terms of requests/second, is almost identical. best, Erik Hetzner ;; Erik Hetzner, California Digital Library ;; gnupg key id: 1024D/01DB07E3 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://rubyforge.org/pipermail/mongrel-users/attachments/20070907/d1198563/attachment.bin
On Sep 07, 2007, at 11:34 pm, Kirk Haines wrote:> Whether all of Rails is actually threadsafe, and whether one''s code > is....that''s another thing, but it''s easy to run Rails in multiple > threads under Mongrel if one wants to play with that.Just out of curiosity, does anyone here use Nitro? I''m going to switch from Rails to Nitro for one of my own projects. I''ve heard it runs happily multi-threaded. Anyone noticed any difference in performance? (Or for that matter, with other frameworks that run multi-threaded like Camping and Merb?) To my knowledge, the only apps big enough for scalability to be a serious issue are all in Rails. Also, this thread has answered the question I''ve wondered lately, namely that MRI can''t context switch if one thread is in a C extension like a database adapter?. Will Rubinius go any way to help this, assuming the adapter is re-entrant? Ashley
> taking them there, alas). Ramaze is also a nice, multithread-safe > framework that merits some attention. It is kind of like Nitro''s > Merb. Inspired by Nitro, but lighter weight, reasonably fast, withnow that MRI1.9 has Coroutines, it might be interseting to check out some of the Lua frameworks/webservers multiplexed on coroutines rather than threads, like Copas / Xavante. er, i guess that would have to wait for anything to run on MRI1.9, nevermind..
On 9/8/07, Ashley Moran <work at ashleymoran.me.uk> wrote:> Just out of curiosity, does anyone here use Nitro? I''m going to > switch from Rails to Nitro for one of my own projects. I''ve heard it > runs happily multi-threaded. Anyone noticed any difference in > performance? (Or for that matter, with other frameworks that run > multi-threaded like Camping and Merb?) To my knowledge, the only > apps big enough for scalability to be a serious issue are all in Rails.I don''t use Nitro, but I have followed its development for a couple years, and have done some benchmarks with it. It is somewhat faster than Rails (though there was a very extensive rewrite that is nearing completion for the 0.50 release, and I have no idea what the rewrite has done to overall performance), and yes, it''s Mongrel handler runs in a multithreaded mode. Actually, every framework except for Rails will Run in that way. Merb, as an example, runs multithreaded, and is one of the fastest Ruby frameworks (in my benchmarks, only IOWA is faster), and it has some projects that do use clusters (I am helping someone setup a small clustered app for demo purposes this weekend, for example). I have written several applications that were designed for clustering and for heavy, multiserver use with IOWA (but the customers never ended up taking them there, alas). Ramaze is also a nice, multithread-safe framework that merits some attention. It is kind of like Nitro''s Merb. Inspired by Nitro, but lighter weight, reasonably fast, with some good ideas in it.> Also, this thread has answered the question I''ve wondered lately, > namely that MRI can''t context switch if one thread is in a C > extension like a database adapter?. Will Rubinius go any way to help > this, assuming the adapter is re-entrant?With Rubinius (or JRuby) it is not the same issue because they are using OS level threads. Kirk Haines
On Sep 08, 2007, at 10:02 pm, Kirk Haines wrote:> Ramaze is also a nice, multithread-safe > framework that merits some attention. It is kind of like Nitro''s > Merb. Inspired by Nitro, but lighter weight, reasonably fast, with > some good ideas in it.That one had passed me by. Thanks for the heads up!> With Rubinius (or JRuby) it is not the same issue because they are > using OS level threads.I appreciate that the issue is different with JRuby, since what matters there is whether the Java libraries are thread-safe (although I saw this week a project to give JRuby access to C libraries using JNI, name escapes me). As for Rubinius, I figured it is in the same position as the Apache web server, where you can''t use the worker MPM if you want to load thread-unsafe modules or modules that depend on thread-unsafe libraries (eg mod_php). And I guess Rubinius still has the option of going the Erlang way of mapping N(requests) green threads onto N (cores) OS threads. Please forgive and wildly incorrect statements I may have made... concurrent programming isn''t something I know much about, mainly because I''ve avoided it due to the mind-boggling complexity of it (well, of threads). Ashley
At Fri, 7 Sep 2007 15:34:49 -0700, "Kirk Haines" <wyhaines at gmail.com> wrote:> Maybe, but it''s irrelevant. The MySQL/Pg/whatever drivers aren''t > written to be aware of Ruby internals.It is my estimation that any application is going to have a lot of places where a ruby thread context switch is possible.> Ezra? His response simplified it. Of course there are situations > where green threads improve performance. They can be quantified very > simply, as I already have. When there are external latencies that are > not blocking the entire process, green threads can capture those > latencies and turn them into additional work. A sleep() call is such > a latency. Any pure ruby code that stays in rubyland and doesn''t go > into an external extension is such a latency. A DB query that spends > most of its time inside of some C library that is external to Ruby is > not such a latency.Here is what I am trying to get at. (a) It is nonsense to say that because ruby threads are not ?real? they will never be as fast as ?real? threads or multiple processes. It is dependent, as I take you to mean, on the balance of your code between code from which context switching can happen & code from which it cannot. (b) *If* you have an application that is threadsafe, you ought to *test* it with multiple threads before you discount the possibility that you will be able to get *comparable* performance out of ruby?s threading than you will from multiple processes, esp. because running multiple threads rather than multiple processes provides some benefits. It is my belief that in general with a threadsafe mongrel app you are going to find that the best system is one that combines multiple processes to take advantage of multiple cores & of preemptive multitasking and multiple threads to take advantage of lower memory consumption and the faster context switching which ruby should provide. Obviously everything depends on (a) your code, (b) your ruby implementation, (c) your os?s scheduler, and (d) the phase of the moon, which is why you ought to test to find out what works best. best, Erik Hetzner -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://rubyforge.org/pipermail/mongrel-users/attachments/20070908/310c053b/attachment.bin
On Sep 08, 2007, at 11:19 pm, Erik Hetzner wrote:> It is my belief that in general with a threadsafe mongrel app you are > going to find that the best system is one that combines multiple > processes to take advantage of multiple cores & of preemptive > multitasking and multiple threads to take advantage of lower memory > consumption and the faster context switching which ruby should > provide.Like I touched on in my last post, this is the strategy that Erlang takes, right? (They seem to have the concurrency problem solved.) My immediate interest here is mainly lower memory consumption. The server (read - dirt cheap PC) I just bought "only" has 1GB RAM, and will only hold 2GB. Simply because I can''t afford to go and buy anything bigger right now, I''d like to know I don''t have to waste RAM running a shedload of 50-200MB processes.> Obviously everything depends on (a) your code, (b) your ruby > implementation, (c) your os?s scheduler, and (d) the phase of the > moonTrue. Lycanthropy is not conducive to good software development. Ashley
On 9/8/07, Ashley Moran <work at ashleymoran.me.uk> wrote:> Like I touched on in my last post, this is the strategy that Erlang > takes, right? (They seem to have the concurrency problem solved.) > My immediate interest here is mainly lower memory consumption. The > server (read - dirt cheap PC) I just bought "only" has 1GB RAM, and > will only hold 2GB. Simply because I can''t afford to go and buy > anything bigger right now, I''d like to know I don''t have to waste RAM > running a shedload of 50-200MB processes.Just as a practical example, I run about 60 separate backend ruby processes on one very modest server with 2GB of RAM, and a bunch more across a few other servers. These 60+ processes represent a cross section of communications/request management technique, all backed by the same basic framework, IOWA. Some are multithreaded with a mod_ruby based handler for communications to the backends (these are the oldest ones). Some use FCGI as the communications conduit, to multithreaded backends. Some run in Mongrel via a mongrel handler, multi-threaded. Some use the Mongrel HTTP parser inside an event based application loop, rendering them effectively single threaded, event based apps. And some of them run behind Swiftiply, some with one node, and some with multiple nodes. On the one machine that has 60 backends on it, each of those represents a separate site. That is, all of the sites are ran off of a single process. They all use the same basic framework -- IOWA -- regardless of the communications model being used. The sites with the greatest throughput capacity are invariably the ones that run an event based communications model -- either the mongrel HTTP parser inside an event loop, or behind Swiftiply. I have sites that, with every page rendered dynamically (from content stored in a database), are capable of handling 285 requests/second with no concurrency, up to about 360/second with a concurrency of 50 (measured using ab). If I switch it to a multithreaded model (mongrel handler), the best that I can do, with no concurrency, is 220 page requests/second, and if I have much concurrency at all, that number quickly drops down towards 150/second. I have two points, here. First, you can run a considerable number of ruby based sites on a single modest server like yours, and they can handle a rather impressive traffic load on a single process, depending on your framework. And second, running the same code in an event based pipeline versus a multithreaded pipeline invariably gives me better peformance in my apps. Kirk Haines
On Sep 09, 2007, at 12:49 am, Kirk Haines wrote:> I have two points, here. First, you can run a considerable number of > ruby based sites on a single modest server like yours, and they can > handle a rather impressive traffic load on a single process, depending > on your framework. And second, running the same code in an event > based pipeline versus a multithreaded pipeline invariably gives me > better peformance in my apps.Interesting! What about the case of long-running requests, eg file uploads, outgoing web service requests or complex calculations? Do you have any of those, and if so, how does it affect the balance of event-based vs multithreaded? Ashley
On 9/8/07, Ashley Moran <work at ashleymoran.me.uk> wrote:> > Interesting! What about the case of long-running requests, eg file > uploads, outgoing web service requests or complex calculations? Do > you have any of those, and if so, how does it affect the balance of > event-based vs multithreaded?Yeah. File uploads -- the most efficient way to deal with these is really to run a separate, dedicated mini-app just for them. If I remember my history correctly, this is actually what originally gave rize to Merb. Ezra needed something fast and lightweight to efficiently handle file uploads. In Ruby, a dedicated file upload handler running event based, though, will still outperform the multithreaded approach. That might make an interesting topic for a blog post sometime soon.... Regarding accessing web services, it all depends on whether those accesses are done in a way that block other threads or not. If they do not, then heavy web service access would be a place where latencies could also be captured. Heavy calculations, on the other hand, are a hands down winner for the single threaded approach. There are no latencies to capture -- it''s CPU all the way, so you just need to bring all the resources available to the process to bear on one problem at a time. Kirk haines
On Sep 09, 2007, at 2:31 am, Kirk Haines wrote:> In Ruby, a dedicated file upload handler running event based, though, > will still outperform the multithreaded approach. That might make an > interesting topic for a blog post sometime soon....Definately!
At Sat, 8 Sep 2007 16:49:58 -0700, "Kirk Haines" <wyhaines at gmail.com> wrote:> I have two points, here. First, you can run a considerable number of > ruby based sites on a single modest server like yours, and they can > handle a rather impressive traffic load on a single process, depending > on your framework. And second, running the same code in an event > based pipeline versus a multithreaded pipeline invariably gives me > better peformance in my apps.Hi Kirk, Thanks so much for your information here. I?m going to have a closer look at your event driven mongrel and see what kind of results I can get out of it for my systems. best, Erik Hetzner -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://rubyforge.org/pipermail/mongrel-users/attachments/20070910/a677a1d0/attachment.bin
At Sat, 8 Sep 2007 16:49:58 -0700, "Kirk Haines" <wyhaines at gmail.com> wrote:> I have two points, here. First, you can run a considerable number of > ruby based sites on a single modest server like yours, and they can > handle a rather impressive traffic load on a single process, depending > on your framework. And second, running the same code in an event > based pipeline versus a multithreaded pipeline invariably gives me > better peformance in my apps.Hi Kirk, Thanks so much for your information here. I?m going to have a closer look at your event driven mongrel and see what kind of results I can get out of it for my systems. best, Erik Hetzner -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://rubyforge.org/pipermail/mongrel-users/attachments/20070910/7ac39b85/attachment-0001.bin
On 9/10/07, Erik Hetzner <erik.hetzner at ucop.edu> wrote:> Thanks so much for your information here. I''m going to have a closer > look at your event driven mongrel and see what kind of results I can > get out of it for my systems.Cool. Feel free to email me if you have any questions. There is a new release Swiftiply and of evented_mongrel coming just as soon as I manage to disentangle myself from client facing work I''m getting buried with. From a scalability standpoint, the new release of evented_mongrel enables it to use epoll on platforms that support that (Linux 2.6.x platforms), instead of select. I''ve been saying for two weeks, now, "This week...." but...hopefully this week I get to the light at the end of the tunnel. ;) Kirk Haines
So I''d say overall the concensus is that multi-threading might be faster, depends on where an apps specific latencies end up (notice the might). Now if I could only come up with a multi-thread safe version of Rails... :) [ha ha] and pure ruby DB interfaces... :) Thank you all! -Roger Belief is good. http://www.google.com/search?q=free+bible -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20070911/dded4681/attachment.html
On Sep 11, 2007, at 15:02 , Roger Pack wrote:> So I''d say overall the concensus is that multi-threading might be > faster, depends on where an apps specific latencies end up (notice > the might). > > Now if I could only come up with a multi-thread safe version of > Rails... :) [ha ha] and pure ruby DB interfaces... :) > > Thank you all! > -RogerMerb + Sequel :) ~Wayne s///g Wayne E. Seguin Sr. Systems Architect & Systems Administrator -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20070911/f21441ff/attachment.html
On Sep 11, 2007, at 8:02 pm, Roger Pack wrote:> pure ruby DB interfaces... :)There''s a postgres-pr gem, but I don''t use it myself. I tell people running Windows they need it, but I always go for the C version myself. If I was in a situation where I had a lot of long-running queries, would I be better with the pure Ruby interface from a concurrency point of view? (I''m assuming that as Ruby VMs improve, this will become true in any case) Ashley
On 9/11/07, Ashley Moran <work at ashleymoran.me.uk> wrote:> There''s a postgres-pr gem, but I don''t use it myself. I tell people > running Windows they need it, but I always go for the C version > myself. If I was in a situation where I had a lot of long-running > queries, would I be better with the pure Ruby interface from a > concurrency point of view? > > (I''m assuming that as Ruby VMs improve, this will become true in any > case)It depends. In _general_, the blocking, C extension based drivers will still net you better performance, currently, becase they are so much faster than the pure ruby drivers. Additionally, because they are typically just interfaces to the database''s own API layer, you know that they are stable and compatible, while the pure ruby implementations may not be as stable. However, if a large number of your queries are very slow, then there may be room to use some of those wasted cycles by using a multithreaded deployment. It''s something that one would really have to benchmark to know for sure for any given use case. Kirk Haines
On Sep 11, 2007, at 9:40 pm, Kirk Haines wrote:> However, if a large number of your queries are very slow, then there > may be room to use some of those wasted cycles by using a > multithreaded deployment. It''s something that one would really have > to benchmark to know for sure for any given use case.How much time do you have to write that blog post you mentioned ;)
Thanks to all for your helpful answers on this question. It''s nice to belong to a mailing list that has smart users (not that others don''t, of course). So overall the answer to my question is that a multi-threaded rails app might well not be significantly faster because say it has 10 threads, if 1 thread is waiting on a long-taking DB query, the other 9 are blocked anyway, so the benefits of a multi-threaded model are moot (so a single-threaded rails is perhaps about as fast as a multi-threaded rails). Bring on Ruby 2.0 :) Further question: would providing the option of having mongrel bind to UNIX sockets perhaps speed it? (i.e. bind to port 3000 and unix socket x). Just a thought. Probably not enough speed difference to matter, but perhaps useful for local processes and simple to implement. Thoughts?> Here''s one question that I think would have an impact on how useful > green threads are for many server apps: > > If you are blocking on something like a database SELECT in one green > thread, will the Ruby interpreter switch to another green thread > while it''s waiting for the response? > > Or does a blocking request block all green threads?>When the flow of execution leaves Ruby and goes into some 3rd party >extension (like that of a DB library), then ruby can''t context switch.>So yes, when one does a DB query that is going through a C extension, >all of the other green threads are blocked, so that latency can''t be >captured.>If it could be, then yes, there would be some benefit from some number >of green threads per process for typical web apps.
On 10/5/07, Roger Pack <rogerpack2005 at gmail.com> wrote:> Thanks to all for your helpful answers on this question. It''s nice to > belong to a mailing list that has smart users (not that others don''t, > of course). > So overall the answer to my question is that a multi-threaded rails > app might well not be significantly faster because say it has 10 > threads, if 1 thread is waiting on a long-taking DB query, the other 9 > are blocked anyway, so the benefits of a multi-threaded model are moot > (so a single-threaded rails is perhaps about as fast as a > multi-threaded rails). Bring on Ruby 2.0 :)Multi-threaded may even be slower, in fact. :) In a lot of the cases, it will definitely be slower. Even on a platform with native threads, multithreading may be no faster than a multiprocess model. It''s not a magic bullet.> Further question: > would providing the option of having mongrel bind to UNIX sockets > perhaps speed it? (i.e. bind to port 3000 and unix socket x). Just a > thought. Probably not enough speed difference to matter, but perhaps > useful for local processes and simple to implement. Thoughts?I used to run all of my IOWA apps via unix sockets. I don''t do that anymore, though. There are situations where it can arguably be useful, but most of the time it just doesn''t matter. Any differences between 127.0.0.1:11111 and /path/to/a/unix/socket are lost in the noise of the rest of the performance affecting load -- they are just too small to matter much. Kirk Haines
On 10/5/07, Roger Pack <rogerpack2005 at gmail.com> wrote:> > > Further question: > would providing the option of having mongrel bind to UNIX sockets > perhaps speed it? (i.e. bind to port 3000 and unix socket x). Just a > thought. Probably not enough speed difference to matter, but perhaps > useful for local processes and simple to implement. Thoughts?Depending on the kernel, you may find Unix sockets to be less performant than localhost TCP. The dominant effect on performance comes from getting the network out of the picture. Beyond that, I''ve often found that unix-domain I/O stacks don''t seem to be as well-tuned as TCP stacks are. And use localhost/tcp instead of an IP address on a network interface. On most kernels, that saves you a trip through the network interface driver. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20071005/22dd81de/attachment-0001.html