We have a method that is just taking a ridiculous amount of time in production: Production: Completed in 84043ms (DB: 35243) | 302 Found [http://x.com/admin/x/ 3262/x] (pid:10052) Dev: Completed in 268ms (DB: 201) | 302 Found [http://c.local/admin/x/3262/ x] (pid:27599) To do this test I dumped the production database, loaded it into my dev environment and performed the identical task on the identical data. Our production environment is an Amazon EC2 instance, which ok is not as fast as my MacPro, but still! What can I do to figure out what is going on here? I assume it is perhaps some sort of db locking? But this happens even when no one else is using the production system. All ideas and tips welcomed! -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
Frederick Cheung
2009-Dec-04 18:55 UTC
Re: Help! incredible timing differences between dev and production
On Dec 4, 6:47 pm, phil <p...-o0Q4Q1ys/oVBDgjK7y7TUQ@public.gmane.org> wrote:> > What can I do to figure out what is going on here? I assume it is > perhaps some sort of db locking? But this happens even when no one > else is using the production system. >I''d start by looking at the difference between explain statements on your local machine and the EC2 instance. If you can narrow it down to one particular query that is a lot slower then that would certainly help. Fred> All ideas and tips welcomed!-- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
phil
2009-Dec-04 20:07 UTC
Re: Help! incredible timing differences between dev and production
What I have done is turned on debugging in a console session on production and then executed the methods. They are blazing fast - at least as fast as on dev! Why would executing them via passenger take so much longer. It is a very consistent thing - every request to this certain method often times out the browser it is so long. Yet... from console it is fast. What would be different about these two (passenger call vs console on same box)? On Dec 4, 7:55 pm, Frederick Cheung <frederick.che...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> On Dec 4, 6:47 pm, phil <p...-o0Q4Q1ys/oVBDgjK7y7TUQ@public.gmane.org> wrote: > > > > > What can I do to figure out what is going on here? I assume it is > > perhaps some sort of db locking? But this happens even when no one > > else is using the production system. > > I''d start by looking at the difference between explain statements on > your local machine and the EC2 instance. If you can narrow it down to > one particular query that is a lot slower then that would certainly > help. > > Fred > > > > > All ideas and tips welcomed!-- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
Niels Meersschaert
2009-Dec-04 20:27 UTC
Re: Re: Help! incredible timing differences between dev and production
I''d imagine in the console you are only looking at the model end of the stack. Is the response in the browser have a large volume to return & lots to render? That will easily account for the difference. On Dec 4, 2009, at 3:07 PM, phil wrote:> What I have done is turned on debugging in a console session on > production and then executed the methods. They are blazing fast - at > least as fast as on dev! Why would executing them via passenger take > so much longer. > > It is a very consistent thing - every request to this certain method > often times out the browser it is so long. Yet... from console it is > fast. What would be different about these two (passenger call vs > console on same box)? > > > On Dec 4, 7:55 pm, Frederick Cheung <frederick.che...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> > wrote: >> On Dec 4, 6:47 pm, phil <p...-o0Q4Q1ys/oVBDgjK7y7TUQ@public.gmane.org> wrote: >> >> >> >>> What can I do to figure out what is going on here? I assume it is >>> perhaps some sort of db locking? But this happens even when no one >>> else is using the production system. >> >> I''d start by looking at the difference between explain statements on >> your local machine and the EC2 instance. If you can narrow it down to >> one particular query that is a lot slower then that would certainly >> help. >> >> Fred >> >> >> >>> All ideas and tips welcomed! > > -- > > You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. > To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en. > >-- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
Colin Law
2009-Dec-04 21:35 UTC
Re: Re: Help! incredible timing differences between dev and production
2009/12/4 phil <phil-o0Q4Q1ys/oVBDgjK7y7TUQ@public.gmane.org>:> What I have done is turned on debugging in a console session on > production and then executed the methods. They are blazing fast - at > least as fast as on dev! Why would executing them via passenger take > so much longer. > > It is a very consistent thing - every request to this certain method > often times out the browser it is so long. Yet... from console it is > fast. What would be different about these two (passenger call vs > console on same box)?If you have narrowed it down to one method you could put some logging in there, logging with the current time, and narrow it down further. Colin -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
Jeffrey L. Taylor
2009-Dec-05 05:48 UTC
Re: Help! incredible timing differences between dev and production
Quoting phil <phil-o0Q4Q1ys/oVBDgjK7y7TUQ@public.gmane.org>:> We have a method that is just taking a ridiculous amount of time in > production: > > Production: > Completed in 84043ms (DB: 35243) | 302 Found [http://x.com/admin/x/ > 3262/x] (pid:10052) > > Dev: > Completed in 268ms (DB: 201) | 302 Found [http://c.local/admin/x/3262/ > x] (pid:27599) > > To do this test I dumped the production database, loaded it into my > dev environment and performed the identical task on the identical > data. > Our production environment is an Amazon EC2 instance, which ok is not > as fast as my MacPro, but still! >What size EC2 instance are you using? And what size is MacPro? A small EC2 instance is the equivalent of a 1GHz Opteron. I''d expect it to be slower than a 2.XGHz Quad-core. And are they running the same DB server. And is the DB server in the same instance? My small EC2 instance is slower than my 1.6GHz single-CPU laptop, but only 10-20%. Both are running Apache, Passenger, and MySQL. I don''t find significant differences between running Webrick and Apache w/ Passenger on my laptop in terms of response times. Jeffrey -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
phil
2009-Dec-05 08:24 UTC
Re: Help! incredible timing differences between dev and production
I think I have found the answer - sweepers! We are using a file_store and when the sweepers run on the ec2 instance it is taking forever. (like 10 seconds to rm a directory). Obviously it is time to revisit the caching! On Dec 5, 6:48 am, "Jeffrey L. Taylor" <r...-f/t7CGFWhwGcvWdFBKKxig@public.gmane.org> wrote:> Quoting phil <p...-o0Q4Q1ys/oVBDgjK7y7TUQ@public.gmane.org>: > > > > > > > We have a method that is just taking a ridiculous amount of time in > > production: > > > Production: > > Completed in 84043ms (DB: 35243) | 302 Found [http://x.com/admin/x/ > > 3262/x] (pid:10052) > > > Dev: > > Completed in 268ms (DB: 201) | 302 Found [http://c.local/admin/x/3262/ > > x] (pid:27599) > > > To do this test I dumped the production database, loaded it into my > > dev environment and performed the identical task on the identical > > data. > > Our production environment is an Amazon EC2 instance, which ok is not > > as fast as my MacPro, but still! > > What size EC2 instance are you using? And what size is MacPro? A small EC2 > instance is the equivalent of a 1GHz Opteron. I''d expect it to be slower than > a 2.XGHz Quad-core. And are they running the same DB server. And is the DB > server in the same instance? My small EC2 instance is slower than my 1.6GHz > single-CPU laptop, but only 10-20%. Both are running Apache, Passenger, and > MySQL. > > I don''t find significant differences between running Webrick and Apache w/ > Passenger on my laptop in terms of response times. > > Jeffrey-- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.