Hey all Im working on improving the speed of my application. Ive taken on board everything that has been said on this forum so far, im using httperf to benchmark and try to improve performance. However, I have some question marks over how best to go about improving performance in certain areas.... With no caching or such like deployed the application runs at an rather slow 35 - 38 req/s. A static file from the server runs at 872 req/s - so im rather far behind that. A basic rails app with render text runs at around 200 req/s. So realistically there is a lot more tuning to be done to get it to a decent level. Im guessing it changes from app to app but if I aim to get near 100 req/s that would improve performance to the point of being much quicker for users? The nature of the application is such that its difficult to do a lot of caching in the traditional sense, is there anyway I can make a custom caching mechanism to cache the content in the way I want? Many thanks in advance Tim ________________________________________________________________________ This e-mail has been scanned for all viruses by Star. The service is powered by MessageLabs. For more information on a proactive anti-virus service working around the clock, around the globe, visit: http://www.star.net.uk ________________________________________________________________________
On Sep 1, 2006, at 4:54 AM, Tim Perrett wrote:> Hey all > > Im working on improving the speed of my application. Ive taken on > board > everything that has been said on this forum so far, im using > httperf to > benchmark and try to improve performance. However, I have some > question > marks over how best to go about improving performance in certain > areas.... > > With no caching or such like deployed the application runs at an > rather slow > 35 - 38 req/s. A static file from the server runs at 872 req/s - so im > rather far behind that. A basic rails app with render text runs at > around > 200 req/s. So realistically there is a lot more tuning to be done > to get it > to a decent level. Im guessing it changes from app to app but if I > aim to > get near 100 req/s that would improve performance to the point of > being much > quicker for users? > > The nature of the application is such that its difficult to do a > lot of > caching in the traditional sense, is there anyway I can make a custom > caching mechanism to cache the content in the way I want? > > Many thanks in advance > > TimNot that it answer''s your question directly but just offer the perspective that http://weblog.textdrive.com/article/73/sencers-caching-plugin-for- textpattern-is-snazzy PHP-MySQL based textpattern is "fast" at ~30 req/sec on a single CPU''ed server. And if we put the 35-38 req/sec in the context of how much you''re serving. Let''s say that text+javascript+images+?? = 125 kilobytes per page, and let''s assume that you''re app is serving 30 req/second. 30req/sec x 125kb page = 3,750 kb/sec = 3.75 MB/sec => 3.75 MB/sec x 60 sec/minute x 60minutes/hour x 24 hours/day = 324,000 MB/day = 324 GB/day So 30 req/sec is 324 GB of network bandwidth out per day; in a 30 day month it''s 9720 GBs or 9.72 TBs or 30Mbps constant. Is that at about where you''ll be? Is this one mongrel process you''re talking about? Now that''s also a different issue from being "much quicker for users", that''s page latency not req/second throughput. - Jason
Actually that math is a bit off because it''s equating a "page view"/ sec to a request/sec. A page might request 20 things. - Jason On Sep 1, 2006, at 5:08 AM, Jason A. Hoffman wrote:> On Sep 1, 2006, at 4:54 AM, Tim Perrett wrote: >> Hey all >> >> Im working on improving the speed of my application. Ive taken on >> board >> everything that has been said on this forum so far, im using >> httperf to >> benchmark and try to improve performance. However, I have some >> question >> marks over how best to go about improving performance in certain >> areas.... >> >> With no caching or such like deployed the application runs at an >> rather slow >> 35 - 38 req/s. A static file from the server runs at 872 req/s - >> so im >> rather far behind that. A basic rails app with render text runs at >> around >> 200 req/s. So realistically there is a lot more tuning to be done >> to get it >> to a decent level. Im guessing it changes from app to app but if I >> aim to >> get near 100 req/s that would improve performance to the point of >> being much >> quicker for users? >> >> The nature of the application is such that its difficult to do a >> lot of >> caching in the traditional sense, is there anyway I can make a custom >> caching mechanism to cache the content in the way I want? >> >> Many thanks in advance >> >> Tim > > Not that it answer''s your question directly but just offer the > perspective that > > http://weblog.textdrive.com/article/73/sencers-caching-plugin-for- > textpattern-is-snazzy > > PHP-MySQL based textpattern is "fast" at ~30 req/sec on a single > CPU''ed server. > > And if we put the 35-38 req/sec in the context of how much you''re > serving. Let''s say that text+javascript+images+?? = 125 kilobytes per > page, and let''s assume that you''re app is serving 30 req/second. > > 30req/sec x 125kb page = 3,750 kb/sec = 3.75 MB/sec > > => 3.75 MB/sec x 60 sec/minute x 60minutes/hour x 24 hours/day > 324,000 MB/day = 324 GB/day > > So 30 req/sec is 324 GB of network bandwidth out per day; in a 30 day > month it''s 9720 GBs or 9.72 TBs or 30Mbps constant. > > Is that at about where you''ll be? Is this one mongrel process you''re > talking about? > > Now that''s also a different issue from being "much quicker for > users", that''s page latency not req/second throughput. > > - Jason
Ok that''s useful!! Its less the bandwidth im worried about (however you do make a very very good point regarding that!) - im more concerned that the application seems really really quick on my internal network, but is quite slow when accessed internally. I just made the assumption that the more requests the application could deliver the quicker the application was, so that''s what I was drawing a connection between that, however am I right in saying that? If im needing to boost performance how might one go about doing it? Cheers Tim -------------------------- Not that it answer''s your question directly but just offer the perspective that http://weblog.textdrive.com/article/73/sencers-caching-plugin-for- textpattern-is-snazzy PHP-MySQL based textpattern is "fast" at ~30 req/sec on a single CPU''ed server. And if we put the 35-38 req/sec in the context of how much you''re serving. Let''s say that text+javascript+images+?? = 125 kilobytes per page, and let''s assume that you''re app is serving 30 req/second. 30req/sec x 125kb page = 3,750 kb/sec = 3.75 MB/sec => 3.75 MB/sec x 60 sec/minute x 60minutes/hour x 24 hours/day 324,000 MB/day = 324 GB/day So 30 req/sec is 324 GB of network bandwidth out per day; in a 30 day month it''s 9720 GBs or 9.72 TBs or 30Mbps constant. Is that at about where you''ll be? Is this one mongrel process you''re talking about? Now that''s also a different issue from being "much quicker for users", that''s page latency not req/second throughput. - Jason ________________________________________________________________________ This e-mail has been scanned for all viruses by Star. The service is powered by MessageLabs. For more information on a proactive anti-virus service working around the clock, around the globe, visit: http://www.star.net.uk ________________________________________________________________________
On 9/1/06, Tim Perrett <tperrett at butlerandtanner.com> wrote:> If im needing to boost performance how might one go about doing it?35-38/second can move a lot of traffic, as has been mentioned, so long as it''s 35-38/second for your dynamic traffic, and your static files are much, much faster. However, when you benchmark 35-38 requests per second, how many concurrent requests are you handing? Can you get 35-38/second with one request at a time, or do you need several concurrent requests to get that speed? How long does it take to actually render a page from your application, from the time the request goes into Rails to the time that it comes out? That latency can make a big difference in the apparent speed of your application, even if your total throughput is good. As for how to speed up page rendering, is there anything that you query from the database that you can cache to avoid db traffic during a hit? That can make a very significant difference. It doesn''t matter if you can render 300 pages per second of plain text if, when you do the required db activity, the db bottlenecks you down to 10% of that. Within your Ruby itself there are some easy things that can make differences. If you use inject() anywhere, benchmark that usage. inject() can be elegant as heck, but it''s also often as slow as it is elegant, and it tends to create a lot of junk objects that have to be garbage collected. It can sometimes be phenomenally slow compared to a solution that doesn''t use it. Do you do any string creation, aggregating from smaller pieces? Use << instead of +. In hash lookups with strings, hash dups the key. If you give it a frozen string it''s faster. If you do a lot of hash lookups this can add up. In general, try to be aware of your frivolous object creation. Ruby isn''t terribly slow with object creation, but it''s still more expensive to use a new object than to reuse an old one, especially when garbage collection is taken into account. But before you spend too much time on any of that, take a look at what you can do to cache your database transactions in order to minimize them. If that is the bottleneck, everything else is a meager drop in the bucket. Kirk Haines
HI~ On Sep 1, 2006, at 7:07 AM, Kirk Haines wrote:> On 9/1/06, Tim Perrett <tperrett at butlerandtanner.com> wrote: > >> If im needing to boost performance how might one go about doing it? > > 35-38/second can move a lot of traffic, as has been mentioned, so long > as it''s 35-38/second for your dynamic traffic, and your static files > are much, much faster. > > However, when you benchmark 35-38 requests per second, how many > concurrent requests are you handing? Can you get 35-38/second with > one request at a time, or do you need several concurrent requests to > get that speed? How long does it take to actually render a page from > your application, from the time the request goes into Rails to the > time that it comes out? > > That latency can make a big difference in the apparent speed of your > application, even if your total throughput is good. > > As for how to speed up page rendering, is there anything that you > query from the database that you can cache to avoid db traffic during > a hit? That can make a very significant difference. It doesn''t > matter if you can render 300 pages per second of plain text if, when > you do the required db activity, the db bottlenecks you down to 10% of > that. > > Within your Ruby itself there are some easy things that can make > differences. > > If you use inject() anywhere, benchmark that usage. inject() can be > elegant as heck, but it''s also often as slow as it is elegant, and it > tends to create a lot of junk objects that have to be garbage > collected. It can sometimes be phenomenally slow compared to a > solution that doesn''t use it. > > Do you do any string creation, aggregating from smaller pieces? Use > << instead of +. > > In hash lookups with strings, hash dups the key. If you give it a > frozen string it''s faster. If you do a lot of hash lookups this can > add up. > > In general, try to be aware of your frivolous object creation. Ruby > isn''t terribly slow with object creation, but it''s still more > expensive to use a new object than to reuse an old one, especially > when garbage collection is taken into account. > > But before you spend too much time on any of that, take a look at what > you can do to cache your database transactions in order to minimize > them. If that is the bottleneck, everything else is a meager drop in > the bucket. > > > Kirk HainesVery good stuff Kirk. I''ll mention a few things from my attempts to speed up rails applications. When you look at a rails app like the one being discussed you need to realize that the view templates are getting rendered every hit if there is no caching. How heavy are your views? Using a ton of link_to and all the other helpers is very nice to work with but can kill your performance. I am workign on an app with almost no caching and tons of ajax. After benchmarking it turned out that rendering the view templates was actually taking around 60% of the total time rails spent servicing a request. By switching a number of link_to''s into <a href>''s and replacing a few other heavy helpers with there normal html equivalents and writing some actual javascript instead of using too many js helpers really cut down on render time. This is one of the most important things to look at in a rails app that cannot use caching. I have also had some great template speed ups from using Stephan Kaes excellent Template Optimizer[1]. Also if you have controller or view helper, or model methods that will get called more then once *during the same request* then use the ||= operator to cache the results like this: def get_something @something ||= big_calculation_to_get_something end This way only the first call to the method will actually do the computation and all the other calls withint the same request will use the cached data in the instance var. Of course you must look at the db too if you are using heavy queries. The development logs can help you see how chatty you are being with the database. Sometimes your view need to render every hit but if you can cache heavy db queries in memory even every 5 or 10 minutes can make a huge difference. Look into memcached or a custom drb caching solution for this depending on how heavily it will be used. Favor memcached if it will play a big role in your app. -Ezra
On Sep 1, 2006, at 8:58 AM, Ezra Zygmuntowicz wrote:> HI~ > > On Sep 1, 2006, at 7:07 AM, Kirk Haines wrote: > >> On 9/1/06, Tim Perrett <tperrett at butlerandtanner.com> wrote: >> >>> If im needing to boost performance how might one go about doing it? >> >> 35-38/second can move a lot of traffic, as has been mentioned, so >> long >> as it''s 35-38/second for your dynamic traffic, and your static files >> are much, much faster. >> >> However, when you benchmark 35-38 requests per second, how many >> concurrent requests are you handing? Can you get 35-38/second with >> one request at a time, or do you need several concurrent requests to >> get that speed? How long does it take to actually render a page from >> your application, from the time the request goes into Rails to the >> time that it comes out? >> >> That latency can make a big difference in the apparent speed of your >> application, even if your total throughput is good. >> >> As for how to speed up page rendering, is there anything that you >> query from the database that you can cache to avoid db traffic during >> a hit? That can make a very significant difference. It doesn''t >> matter if you can render 300 pages per second of plain text if, when >> you do the required db activity, the db bottlenecks you down to >> 10% of >> that. >> >> Within your Ruby itself there are some easy things that can make >> differences. >> >> If you use inject() anywhere, benchmark that usage. inject() can be >> elegant as heck, but it''s also often as slow as it is elegant, and it >> tends to create a lot of junk objects that have to be garbage >> collected. It can sometimes be phenomenally slow compared to a >> solution that doesn''t use it. >> >> Do you do any string creation, aggregating from smaller pieces? Use >> << instead of +. >> >> In hash lookups with strings, hash dups the key. If you give it a >> frozen string it''s faster. If you do a lot of hash lookups this can >> add up. >> >> In general, try to be aware of your frivolous object creation. Ruby >> isn''t terribly slow with object creation, but it''s still more >> expensive to use a new object than to reuse an old one, especially >> when garbage collection is taken into account. >> >> But before you spend too much time on any of that, take a look at >> what >> you can do to cache your database transactions in order to minimize >> them. If that is the bottleneck, everything else is a meager drop in >> the bucket. >> >> >> Kirk Haines > > > Very good stuff Kirk. I''ll mention a few things from my attempts to > speed up rails applications. When you look at a rails app like the > one being discussed you need to realize that the view templates are > getting rendered every hit if there is no caching. How heavy are your > views? Using a ton of link_to and all the other helpers is very nice > to work with but can kill your performance. I am workign on an app > with almost no caching and tons of ajax. After benchmarking it turned > out that rendering the view templates was actually taking around 60% > of the total time rails spent servicing a request. By switching a > number of link_to''s into <a href>''s and replacing a few other heavy > helpers with there normal html equivalents and writing some actual > javascript instead of using too many js helpers really cut down on > render time. This is one of the most important things to look at in a > rails app that cannot use caching. I have also had some great > template speed ups from using Stephan Kaes excellent Template > Optimizer[1].Gah! Sorry forgot to include a link in the last email: [1] http://railsexpress.de/blog/articles/2006/08/15/rails-template- optimizer-beta-test#comments -Ezra
On Fri, 2006-09-01 at 12:54 +0100, Tim Perrett wrote:> Hey all > > Im working on improving the speed of my application. Ive taken on board > everything that has been said on this forum so far, im using httperf to > benchmark and try to improve performance. However, I have some question > marks over how best to go about improving performance in certain areas.... > > With no caching or such like deployed the application runs at an rather slow > 35 - 38 req/s. A static file from the server runs at 872 req/s - so im > rather far behind that. A basic rails app with render text runs at around > 200 req/s. So realistically there is a lot more tuning to be done to get it > to a decent level. Im guessing it changes from app to app but if I aim to > get near 100 req/s that would improve performance to the point of being much > quicker for users? >First, 35-38 isn''t that bad. If you''ve already tried tuning it and this is the best you get then you''re probably the tuning density of the app. Only a redesign or more hardware will really make it faster. In other words, you have to move from focusing on little stuff and now look at big stuff: design, deployment, load balancers, architecture, etc. However, have you asked the users? It might be that 35-38 is just fine. It might be that you''ll never make it fast enough. What you have to do is get an idea of people''s feeling of the speed of the app, and then see what you can do to just make them *think* it''s faster. This gets into more messing with human perception than measuring raw performance metrics, but it''s also a very effective way of making things "faster" without changing much about your app. A lot of this approach involves getting user feedback, coming up with ways you think will improve their impressions, doing the changes and measuring them, then going back for more feedback. You''ll probably stumble on a few simple little changes that will make things seem tons faster. -- Zed A. Shaw http://www.zedshaw.com/ http://mongrel.rubyforge.org/ http://www.lingr.com/room/3yXhqKbfPy8 -- Come get help.
You can also send model objects over to the page you are rendering as JSON and work directly from them, and only post when you need to. It is going to mean a lot of JavaScript work and something like the Unobtrusive Javascripts plugin can help you make it a less painful experience. Expect to spend more time debugging, though ... -- G.