Bryan Migliorisi
2013-Jun-27 16:11 UTC
Asset pre-compilation unbearably slow: Have we outgrown asset pipeline?
Our application is a very large (read: "enterprise") Single Page Application. Even though our app is 99% client side with a Java RESTful API, we built the web application on top of rails because the Asset Pipeline seemed like a a great fit for us. And until recently, it was. Our application has several hundred CoffeeScript and SASS files in it (as we''ve broken our code down in to many smaller pieces for reusability). We deploy to Heroku, who automatically compiles assets for us. It used to be pretty fast. Then it was a minute. Then 2. Then 5. Now, we have apparently hit the upper limit that Heroku is willing to wait for assets to compile (15 mins). Pre-compiling the assets on a developers machine is possible but its error prone and doesnt solve the problem of our compilation process taking forever. I notice that even some of our smaller CoffeeScript files (~50 lines) take 250ms to compile. This quickly adds up with the amount of files we have. On my i7 MBA, assets:precompile:primary takes around 5 minutes and assets:precompile over 10 minutes. I think (but not sure) the biggest problem is that it appears that a new nodejs process is started for every CoffeeScript file and each start takes a bit of time before the file is even processed. We are on the latest Rails 3.2.x. Is there a better way? Is there any tuning I can do for improving the speed of compilation? -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To view this discussion on the web visit https://groups.google.com/d/msgid/rubyonrails-talk/352dc6b7-fae9-4f70-8d08-101c901852d4%40googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Jim Ruther Nill
2013-Jun-28 00:29 UTC
Re: Asset pre-compilation unbearably slow: Have we outgrown asset pipeline?
On Fri, Jun 28, 2013 at 12:11 AM, Bryan Migliorisi <bryan-jehn3jYJiseakBO8gow8eQ@public.gmane.org>wrote:> Our application is a very large (read: "enterprise") Single Page > Application. > > Even though our app is 99% client side with a Java RESTful API, we built > the web application on top of rails because the Asset Pipeline seemed like > a a great fit for us. And until recently, it was. > > Our application has several hundred CoffeeScript and SASS files in it (as > we''ve broken our code down in to many smaller pieces for reusability). > > We deploy to Heroku, who automatically compiles assets for us. It used to > be pretty fast. Then it was a minute. Then 2. Then 5. Now, we have > apparently hit the upper limit that Heroku is willing to wait for assets to > compile (15 mins). > > Pre-compiling the assets on a developers machine is possible but its error > prone and doesnt solve the problem of our compilation process taking > forever. > > I notice that even some of our smaller CoffeeScript files (~50 lines) take > 250ms to compile. This quickly adds up with the amount of files we have. > > On my i7 MBA, assets:precompile:primary takes around 5 minutes and > assets:precompile over 10 minutes. I think (but not sure) the biggest > problem is that it appears that a new nodejs process is started for every > CoffeeScript file and each start takes a bit of time before the file is > even processed. > > We are on the latest Rails 3.2.x. Is there a better way? Is there any > tuning I can do for improving the speed of compilation? >try looking at https://github.com/ndbroadbent/turbo-sprockets-rails3> -- > You received this message because you are subscribed to the Google Groups > "Ruby on Rails: Talk" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > To view this discussion on the web visit > https://groups.google.com/d/msgid/rubyonrails-talk/352dc6b7-fae9-4f70-8d08-101c901852d4%40googlegroups.com > . > For more options, visit https://groups.google.com/groups/opt_out. > > >-- ------------------------------------------------------------- visit my blog at http://jimlabs.heroku.com -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To view this discussion on the web visit https://groups.google.com/d/msgid/rubyonrails-talk/CAJ8y7VfgdNUh44rUnRJSA%3D_BXvpyP6ikOhToV3FM7O2C_z7yJg%40mail.gmail.com. For more options, visit https://groups.google.com/groups/opt_out.
Matt Jones
2013-Jun-28 17:32 UTC
Re: Asset pre-compilation unbearably slow: Have we outgrown asset pipeline?
On Thursday, 27 June 2013 09:11:44 UTC-7, Bryan Migliorisi wrote:> > Our application is a very large (read: "enterprise") Single Page > Application. > > Even though our app is 99% client side with a Java RESTful API, we built > the web application on top of rails because the Asset Pipeline seemed like > a a great fit for us. And until recently, it was. > > Our application has several hundred CoffeeScript and SASS files in it (as > we''ve broken our code down in to many smaller pieces for reusability). > > We deploy to Heroku, who automatically compiles assets for us. It used to > be pretty fast. Then it was a minute. Then 2. Then 5. Now, we have > apparently hit the upper limit that Heroku is willing to wait for assets to > compile (15 mins). > > Pre-compiling the assets on a developers machine is possible but its error > prone and doesnt solve the problem of our compilation process taking > forever. > > I notice that even some of our smaller CoffeeScript files (~50 lines) take > 250ms to compile. This quickly adds up with the amount of files we have. > > On my i7 MBA, assets:precompile:primary takes around 5 minutes and > assets:precompile over 10 minutes. I think (but not sure) the biggest > problem is that it appears that a new nodejs process is started for every > CoffeeScript file and each start takes a bit of time before the file is > even processed. > > We are on the latest Rails 3.2.x. Is there a better way? Is there any > tuning I can do for improving the speed of compilation? >This might help, if you''re deploying to Heroku mostly with assets unchanged: http://www.thirdprestige.com/posts/how-7-lines-of-ruby-will-speed-up-your-heroku-deploys-10x -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To view this discussion on the web visit https://groups.google.com/d/msgid/rubyonrails-talk/cd6b5718-6b8b-4b1c-a178-d932d61eccd7%40googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Bryan Migliorisi
2013-Jun-28 17:49 UTC
Re: Asset pre-compilation unbearably slow: Have we outgrown asset pipeline?
Thanks for the suggestions. I actually set up the turbo-sprockets-rails3 gem and buildpack and the difference is night and day. The first build after switching took only 4 minutes which is way better than the 15+ it was taking before. Consequent builds are taking 30 seconds. Amazing. Im guessing it uses a similar approach under the hood to what this blog post is proposing, but I have not had the time to dig deeper into the code yet. Thanks for the help guys --Bryan On Friday, June 28, 2013 1:32:32 PM UTC-4, Matt Jones wrote:> > > > On Thursday, 27 June 2013 09:11:44 UTC-7, Bryan Migliorisi wrote: >> >> Our application is a very large (read: "enterprise") Single Page >> Application. >> >> Even though our app is 99% client side with a Java RESTful API, we built >> the web application on top of rails because the Asset Pipeline seemed like >> a a great fit for us. And until recently, it was. >> >> Our application has several hundred CoffeeScript and SASS files in it (as >> we''ve broken our code down in to many smaller pieces for reusability). >> >> We deploy to Heroku, who automatically compiles assets for us. It used to >> be pretty fast. Then it was a minute. Then 2. Then 5. Now, we have >> apparently hit the upper limit that Heroku is willing to wait for assets to >> compile (15 mins). >> >> Pre-compiling the assets on a developers machine is possible but its >> error prone and doesnt solve the problem of our compilation process taking >> forever. >> >> I notice that even some of our smaller CoffeeScript files (~50 lines) >> take 250ms to compile. This quickly adds up with the amount of files we >> have. >> >> On my i7 MBA, assets:precompile:primary takes around 5 minutes and >> assets:precompile over 10 minutes. I think (but not sure) the biggest >> problem is that it appears that a new nodejs process is started for every >> CoffeeScript file and each start takes a bit of time before the file is >> even processed. >> >> We are on the latest Rails 3.2.x. Is there a better way? Is there any >> tuning I can do for improving the speed of compilation? >> > > This might help, if you''re deploying to Heroku mostly with assets > unchanged: > > > http://www.thirdprestige.com/posts/how-7-lines-of-ruby-will-speed-up-your-heroku-deploys-10x > >-- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To view this discussion on the web visit https://groups.google.com/d/msgid/rubyonrails-talk/01d7247e-f6b2-49fc-8a13-3395ac48a44c%40googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.