Hi, my video sharing website (politube.org) that I developed with rails, runs on lighttpd/fcgi (I know it is obsolete by now) and is severely under load and regularly blocks because the lighttpd process chews up 50% of RAM (1GB). So now I want to move the system to a new server with 8GB, 64bit , Quad core processor (intel i7 920) but I am not sure if hyperthreading will really do anything better. Even if I switched to mongrel, would it be able to leverage the Quadcore architecture? Any advice would be greatly appreciated. Kuros -- Posted via http://www.ruby-forum.com/.
Hi Kuros, One thing that could certainly help is moving to a apache/nginx + passenger setup. That would improve your memory usage. But keep in mind that if you rely on ruby threads on your application, ruby won''t take advantage of your QuadCore processor. Only if you spawn multiple processes. But again, this depends on your application and how it works. That being said, I wrote a post a whlie ago explaining a bit how ruby handles threads (in 1.8, 1.9 and in jruby). It might be useful as well. Cheers, Leonardo Borges www.leonardoborges.com On Jul 2, 10:34 am, Kuros Yalpani <rails-mailing-l...-ARtvInVfO7ksV2N9l4h3zg@public.gmane.org> wrote:> Hi, my video sharing website (politube.org) that I developed with rails, > runs on lighttpd/fcgi (I know it is obsolete by now) and is severely > under load and regularly blocks because the lighttpd process chews up > 50% of RAM (1GB). > > So now I want to move the system to a new server with 8GB, 64bit , > Quad core processor (intel i7 920) but I am not sure if hyperthreading > will really do anything better. Even if I switched to mongrel, would it > be able to leverage the Quadcore architecture? > > Any advice would be greatly appreciated. > > Kuros > -- > Posted viahttp://www.ruby-forum.com/.
Kuros Yalpani wrote:> Hi, my video sharing website (politube.org) that I developed with rails, > runs on lighttpd/fcgi (I know it is obsolete by now) and is severely > under load and regularly blocks because the lighttpd process chews up > 50% of RAM (1GB). > > So now I want to move the system to a new server with 8GB, 64bit , > Quad core processor (intel i7 920) but I am not sure if hyperthreading > will really do anything better. Even if I switched to mongrel, would it > be able to leverage the Quadcore architecture?Please note that hyperthreading (which is a technique for splitting up one procesor into two virtual processors) is not the same as application threading. Whether hyperthreading will be beneficial is highly dependent on your workload. Benchmarks for your application are your friend. In general, I would say, hyperthreading will be beneficial if you have a workload that is a composite of two distinct and mutually exclusive loads, such as video processing on the one hand and I/O on the other. If the loads are not mutually exclusive then you will have a lot more cache misses and hyperthreading may actually be detrimental. -- Roderick van Domburg http://www.nedforce.com -- Posted via http://www.ruby-forum.com/.
Roderick van Domburg wrote:> Kuros Yalpani wrote: >> Hi, my video sharing website (politube.org) that I developed with rails, >> runs on lighttpd/fcgi (I know it is obsolete by now) and is severely >> under load and regularly blocks because the lighttpd process chews up >> 50% of RAM (1GB). >> >> So now I want to move the system to a new server with 8GB, 64bit , >> Quad core processor (intel i7 920) but I am not sure if hyperthreading >> will really do anything better. Even if I switched to mongrel, would it >> be able to leverage the Quadcore architecture? > > Please note that hyperthreading (which is a technique for splitting up > one procesor into two virtual processors) is not the same as application > threading. > > Whether hyperthreading will be beneficial is highly dependent on your > workload. Benchmarks for your application are your friend. > > In general, I would say, hyperthreading will be beneficial if you have a > workload that is a composite of two distinct and mutually exclusive > loads, such as video processing on the one hand and I/O on the other. If > the loads are not mutually exclusive then you will have a lot more cache > misses and hyperthreading may actually be detrimental. > > -- > Roderick van Domburg > http://www.nedforce.comactually, this would presumably answer my question, since if hyperthreading is emulating multiple cpu''s then the multiprocess architecture of fcgi would benefit from that. However, I am not sure on how load "mutual exclusivity" affects cache usage. Do you mean a kind of instruction cache? Any Idea on how mongrel vs lighttpd behaves? thanks Kuros -- Posted via http://www.ruby-forum.com/.
i found thin seems to run faster then mongrel in my app. just my 2 cents. nayeem 2009/7/2 Kuros Yalpani <rails-mailing-list-ARtvInVfO7ksV2N9l4h3zg@public.gmane.org>> > Roderick van Domburg wrote: > > Kuros Yalpani wrote: > >> Hi, my video sharing website (politube.org) that I developed with > rails, > >> runs on lighttpd/fcgi (I know it is obsolete by now) and is severely > >> under load and regularly blocks because the lighttpd process chews up > >> 50% of RAM (1GB). > >> > >> So now I want to move the system to a new server with 8GB, 64bit , > >> Quad core processor (intel i7 920) but I am not sure if hyperthreading > >> will really do anything better. Even if I switched to mongrel, would it > >> be able to leverage the Quadcore architecture? > > > > Please note that hyperthreading (which is a technique for splitting up > > one procesor into two virtual processors) is not the same as application > > threading. > > > > Whether hyperthreading will be beneficial is highly dependent on your > > workload. Benchmarks for your application are your friend. > > > > In general, I would say, hyperthreading will be beneficial if you have a > > workload that is a composite of two distinct and mutually exclusive > > loads, such as video processing on the one hand and I/O on the other. If > > the loads are not mutually exclusive then you will have a lot more cache > > misses and hyperthreading may actually be detrimental. > > > > -- > > Roderick van Domburg > > http://www.nedforce.com > > actually, this would presumably answer my question, since if > hyperthreading is emulating multiple cpu''s then the multiprocess > architecture of fcgi would benefit from that. However, I am not sure on > how load "mutual exclusivity" affects cache usage. Do you mean a kind of > instruction cache? Any Idea on how mongrel vs lighttpd behaves? > > thanks > > Kuros > -- > Posted via http://www.ruby-forum.com/. > > > >-- cashflowclublondon.co.uk ("`-''''-/").___..--''''"`-._ `6_ 6 ) `-. ( ).`-.__.`) (_Y_.)'' ._ ) `._ `. ``-..-'' _..`--''_..-_/ /--''_.'' ,'' (il),-'''' (li),'' ((!.-'' . --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Leonardo Borges wrote:> Hi Kuros, > One thing that could certainly help is moving to a apache/nginx + > passenger setup. That would improve your memory usage. > > But keep in mind that if you rely on ruby threads on your application, > ruby won''t take advantage of your QuadCore processor. Only if you > spawn multiple processes. But again, this depends on your application > and how it works. > > That being said, I wrote a post a whlie ago explaining a bit how ruby > handles threads (in 1.8, 1.9 and in jruby). It might be useful as > well. > > Cheers, > Leonardo Borges > www.leonardoborges.com > > On Jul 2, 10:34�am, Kuros Yalpani <rails-mailing-l...@andreas-s.net>well, I think it is unclear exactly how Quadcore distributes, on the one hand, in Wikipedia, it is defined as Hyperthreading, with the explanation that there are several virtual processors (which would satisfy my requirements) but at the same time, wikipedia defines Hyperthreading as SMT (simultaneous Multithreading) and if you follow that link, SMT is described as supporting several parallel THREADS (implying that all within the same process). In case of the latter, I guess a Quadcore setup with out of the box rails wouldn''t really help. You suggest BTW apache/nginx, which I never used before, any ideas on how mongrail behaves under load? thanks Kuros -- Posted via http://www.ruby-forum.com/.
Developer In London wrote:> i found thin seems to run faster then mongrel in my app. just my 2 > cents. > > nayeemthanks, are you using a multi cpu (dual/quad) hardware? Kuros> > 2009/7/2 Kuros Yalpani <rails-mailing-list-ARtvInVfO7ksV2N9l4h3zg@public.gmane.org> > >> >> Quad core processor (intel i7 920) but I am not sure if hyperthreading >> > In general, I would say, hyperthreading will be beneficial if you have a >> hyperthreading is emulating multiple cpu''s then the multiprocess >> > >> > > > -- > cashflowclublondon.co.uk > > ("`-''''-/").___..--''''"`-._ > `6_ 6 ) `-. ( ).`-.__.`) > (_Y_.)'' ._ ) `._ `. ``-..-'' > _..`--''_..-_/ /--''_.'' ,'' > (il),-'''' (li),'' ((!.-'' > .-- Posted via http://www.ruby-forum.com/.
Leonardo Borges wrote:> Hi Kuros, > One thing that could certainly help is moving to a apache/nginx + > passenger setup. That would improve your memory usage. > > But keep in mind that if you rely on ruby threads on your application, > ruby won''t take advantage of your QuadCore processor. Only if you > spawn multiple processes. But again, this depends on your application > and how it works. > > That being said, I wrote a post a whlie ago explaining a bit how ruby > handles threads (in 1.8, 1.9 and in jruby). It might be useful as > well. > > Cheers, > Leonardo Borges > www.leonardoborges.com > > On Jul 2, 10:34�am, Kuros Yalpani <rails-mailing-l...@andreas-s.net>Incidentally, I checked out apache/nginx and that seems also interesting to follow, thanks for pointing it out. I looked for you post (above) but couldn''t find it. Kuros -- Posted via http://www.ruby-forum.com/.
Kuros Yalpani wrote:> actually, this would presumably answer my question, since if > hyperthreading is emulating multiple cpu''s then the multiprocess > architecture of fcgi would benefit from that. However, I am not sure on > how load "mutual exclusivity" affects cache usage. Do you mean a kind of > instruction cache? Any Idea on how mongrel vs lighttpd behaves?Sure, multi-process architectures can leverage SMP architectures. But my point is: be aware that having two equal processes run on two HTT CPU''s (so actually a single CPU) will be detrimental to performance, generally. The same code will be running on the same physical CPU components and that needs to run serially, with all of the cache and context issues that brings with it. I prefer a cluster of Passengers, Mongrels or Thins (depending on your performance profile) over FastCGI because I find it to be more stable and easier to manage. I also find the Apache / Nginx / Lighttpd discussion to be a bit of a bike-shed when it comes to serving dynamic Rails requests. The performance differences are not significant except for static assets or when you''re pressed for RAM. All these deployment options are process-based. Apache can run multi-threaded too. I don''t have any production experience with that. -- Roderick van Domburg http://www.nedforce.com -- Posted via http://www.ruby-forum.com/.