I''m working on a pretty large (and still growing) project and the bigger it gets the slower development mode becomes, but runs really fast in production mode. So I assume the slowdown at the beginning of each request is caused by the fact that it reloads everything. It currently has about 85 models, 75 controllers, and 11k Code LOC. It''s becoming really painful. For instance, I''m working on a page that involves a lot of ajax calls. One call shows the following in the development.log: Completed in 0.38060 (2 reqs/sec) | Rendering: 0.02327 (6%) | DB: 0.06884 (18%) But firebug timed it at 5.38 seconds. :( So that means reloading my project (+ I guess javascript and firefox overhead, but all it''s doing is validating values on a form and enabling/disabling a button) added an additional 5 seconds?? And it keeps getting slower over time if I don''t restart the server often. And I''m on an a MacBookPro with 2 gig of ram. Some of the other developers here aren''t quite as lucky as me and it''s even more painful for them. At this point it''s almost making more sense to run with cache_classes true and just restart mongrel when needed. I even considered writing a program to monitor files in my project and restart the server when one changes... another thought was to try and split up the app into separate smaller apps and share things between them, however there''s so much shared logic and model relationships, that I don''t think it would be feasible. Is there anything else I can do to optimize the development environment? Is this a common thing with large rails projects or could there be something I''m doing wrong? What could I do to try and track down the slowness in development? -- Posted via http://www.ruby-forum.com/. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Philip Hallstrom
2007-Aug-29 16:49 UTC
Re: painfully slow development mode on large project
> I''m working on a pretty large (and still growing) project and the bigger > it gets the slower development mode becomes, but runs really fast in > production mode. So I assume the slowdown at the beginning of each > request is caused by the fact that it reloads everything. It currently > has about 85 models, 75 controllers, and 11k Code LOC. > > It''s becoming really painful. For instance, I''m working on a page that > involves a lot of ajax calls. One call shows the following in the > development.log: > > Completed in 0.38060 (2 reqs/sec) | Rendering: 0.02327 (6%) | DB: > 0.06884 (18%) > > But firebug timed it at 5.38 seconds. :( So that means reloading myWhat web server? It would seem that Rails is done with the request pretty quickly... so the extra time is coming from somewhere else. Any chance you''re using webrick? I could see that happening with a lot of ajax as webrick can only handle one request at a time... Our app is about 43,000 lines and has 832 classes and it''s not that slow... we''re running litespeed for development...> project (+ I guess javascript and firefox overhead, but all it''s doing > is validating values on a form and enabling/disabling a button) added an > additional 5 seconds?? And it keeps getting slower over time if I don''t > restart the server often. And I''m on an a MacBookPro with 2 gig of ram. > Some of the other developers here aren''t quite as lucky as me and it''s > even more painful for them. > > At this point it''s almost making more sense to run with cache_classes > true and just restart mongrel when needed. I even considered writing a > program to monitor files in my project and restart the server when one > changes... another thought was to try and split up the app into separate > smaller apps and share things between them, however there''s so much > shared logic and model relationships, that I don''t think it would be > feasible. > > Is there anything else I can do to optimize the development environment? > Is this a common thing with large rails projects or could there be > something I''m doing wrong? What could I do to try and track down the > slowness in development? > -- > Posted via http://www.ruby-forum.com/. > > > >--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Philip Hallstrom wrote:> Any chance you''re using webrick? I could see that happening with a lot > of > ajax as webrick can only handle one request at a time...I''m using mongrel. Regular pages take about 2-3 seconds longer to load then what is shown in development.log. The slow down occurs at the beginning of the request before anything even shows up in the log. It''s like I hit a link, count to 3, then BAM lots of stuff shows up in log and page comes in. On pages that might do multiple ajax calls at the same time, it does tend to take longer. Is mongrel able to handle more then one request at a time?> Our app is about 43,000 lines and has 832 classes and it''s not that > slow... we''re running litespeed for development...How much of a delay for each request do you experience in development mode? -- Posted via http://www.ruby-forum.com/. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Tony Buser wrote:> Philip Hallstrom wrote: > >> Any chance you''re using webrick? I could see that happening with a lot >> of >> ajax as webrick can only handle one request at a time... >> > > I''m using mongrel. Regular pages take about 2-3 seconds longer to load > then what is shown in development.log. The slow down occurs at the > beginning of the request before anything even shows up in the log. It''s > like I hit a link, count to 3, then BAM lots of stuff shows up in log > and page comes in. On pages that might do multiple ajax calls at the > same time, it does tend to take longer. Is mongrel able to handle more > then one request at a time? >Not exactly (the Rails processing can''t be done concurrently, but serving files in <rails_app>/public can). If there is such a delay with near 0 CPU usage, you may want to use strace to see what syscall Mongrel is waiting the result for, it might be a DNS request, a file opening, ... This should point you in the right direction. Lionel --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Faisal N Jawdat
2007-Aug-29 18:04 UTC
Re: painfully slow development mode on large project
As it happens, Josh Goebel just released a plugin to deal with this very problem: http://svn.techno-weenie.net/projects/plugins/ dev_mode_performance_fixes/> Why don''t we only reload the files when they have actually > changed? Store the file names and their last modified time and > then stat all the files before every request... if any are newer, > kill the objects they originally defined and let the Dependency > auto-loading code reload the objects from disk.People are reporting anywhere between "4x" and "almost as fast as production" speed improvements. -faisal --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Faisal N Jawdat wrote:> As it happens, Josh Goebel just released a plugin to deal with this > very problem: > > http://svn.techno-weenie.net/projects/plugins/ > dev_mode_performance_fixes/ > >It catched my eyes too, but I have a project with more LOC, models and controllers than Tony''s project and webrick on a 1.06GHz Centrino Duo is fast enough for me (nearly all actions take less than 1s and most are around 0.3s, 3-4 whole seconds seems odd to me unless he uses something like a Pentium MMX or less for the server), so I don''t think his problem is class reloading (or he must have a pretty peculiar code). Won''t hurt to try though. Lionel --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Philip Hallstrom
2007-Aug-29 18:50 UTC
Re: painfully slow development mode on large project
> Philip Hallstrom wrote: >> Any chance you''re using webrick? I could see that happening with a lot >> of >> ajax as webrick can only handle one request at a time... > > I''m using mongrel. Regular pages take about 2-3 seconds longer to load > then what is shown in development.log. The slow down occurs at the > beginning of the request before anything even shows up in the log. It''s > like I hit a link, count to 3, then BAM lots of stuff shows up in log > and page comes in. On pages that might do multiple ajax calls at the > same time, it does tend to take longer. Is mongrel able to handle more > then one request at a time?One Rails request at a time. Static is threaded...>> Our app is about 43,000 lines and has 832 classes and it''s not that >> slow... we''re running litespeed for development... > > How much of a delay for each request do you experience in development > mode?Hardly any. Obviously it takes longer to load since it''s dev and we''re not caching anything, but it still comes up... Someone else mentioned DNS... that might be worth looking into as well. -philip --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Faisal N Jawdat wrote:> http://svn.techno-weenie.net/projects/plugins/ > dev_mode_performance_fixes/ > > People are reporting anywhere between "4x" and "almost as fast as > production" speed improvements.Great googly moogly!! I owe you and the guy that wrote that a beer... actually I owe you 10 beers because that plugin made my development mode at least 10x faster! It''s like night and day. Practically as fast as production mode. I need to do some more tests to see if it breaks anything, but so far everything is passing and working great. Amazing, why on earth doesn''t rails do this by default? I''m blown away... there must be some downside to this. Philip: I really would like to know what you''re doing differently then I am that would allow a project as large as yours to run so fast. I suppose it''s possible I''m doing something bad in a class somewhere that causes the slowness, but I''m having a hell of a time tracking it down. I tried running it through strace on a linux box, but there doesn''t appear to be any one spot where it sits waiting for something. I do see a TON of calls trying to stat files that don''t exist (-1 ENOENT (No such file or directory)) that has me confused... but I think it is rails trying to autoload files and looking in every possible place it could be? -- Posted via http://www.ruby-forum.com/. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---