I''m doing a migration that needs to go through all rows in a DB table and update then. It takes forever and I''ve narrowed it down to Rails'' own .save() call. If I do all the update without saving, it completes in a few seconds. With the Active Record save, 60000 rows take 15 minutes to update. Is there a way for me to bypass that and write the values directly to the database? -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
On Wed, Nov 18, 2009 at 10:43 AM, helzer <amir.helzer-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> I''m doing a migration that needs to go through all rows in a DB table > and update then. > > It takes forever and I''ve narrowed it down to Rails'' own .save() call. > If I do all the update without saving, it completes in a few seconds. > With the Active Record save, 60000 rows take 15 minutes to update. > > Is there a way for me to bypass that and write the values directly to > the database? > >Where''s the code that you''re using to do the migration? -Conrad> -- > > You received this message because you are subscribed to the Google Groups > "Ruby on Rails: Talk" group. > To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > To unsubscribe from this group, send email to > rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org<rubyonrails-talk%2Bunsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org> > . > For more options, visit this group at > http://groups.google.com/group/rubyonrails-talk?hl=en. > > >-- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
Table.find(:all).each { |e| e.save }
I have some processing per entry, but even if I remove it all and just
do the save, it takes forever.
If I return everything else and just comment out e.save it completes
quickly.
What puzzles me is that the CPU is running 100%, 95% Ruby and only 5%
MySQL, so the actual table updates are not the problem. Active Record
is just loading it.
Amir
--
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en.
Do you have any on_save callbacks on the table''s model? Those are
executing on each run. Also, all your validations are running as well with the
save. You can ignore validations by altering your code to:
Table.find(:all).each {|e| e.save(false)}
However, your callbacks will still execute and if you have any observers, those
get called as well.
If any of those can''t be skipped with a conditional, then you can
always run your updates with raw sql.
On Nov 18, 2009, at 2:17 PM, helzer wrote:
> Table.find(:all).each { |e| e.save }
>
> I have some processing per entry, but even if I remove it all and just
> do the save, it takes forever.
>
> If I return everything else and just comment out e.save it completes
> quickly.
>
> What puzzles me is that the CPU is running 100%, 95% Ruby and only 5%
> MySQL, so the actual table updates are not the problem. Active Record
> is just loading it.
>
> Amir
>
> --
>
> You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
> To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en.
helzer a écrit, le 11/18/2009 07:43 PM :> I''m doing a migration that needs to go through all rows in a DB table > and update then. > > It takes forever and I''ve narrowed it down to Rails'' own .save() call. > If I do all the update without saving, it completes in a few seconds. > With the Active Record save, 60000 rows take 15 minutes to update. > > Is there a way for me to bypass that and write the values directly to > the database? >YourModel.connection.execute("your UPDATE in SQL here") Lionel -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
On Wed, Nov 18, 2009 at 11:17 AM, helzer <amir.helzer-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> Table.find(:all).each { |e| e.save } > >After thinking about what you''re trying to do, it might be much faster to simply perform the update using SQL. Then you''ll avoid AR all together. Otherwise, you might be interested in using something ar-extensions which is great for bulk updates. -Conrad> I have some processing per entry, but even if I remove it all and just > do the save, it takes forever. > > If I return everything else and just comment out e.save it completes > quickly. > > What puzzles me is that the CPU is running 100%, 95% Ruby and only 5% > MySQL, so the actual table updates are not the problem. Active Record > is just loading it. > > Amir > > -- > > You received this message because you are subscribed to the Google Groups > "Ruby on Rails: Talk" group. > To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > To unsubscribe from this group, send email to > rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org<rubyonrails-talk%2Bunsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org> > . > For more options, visit this group at > http://groups.google.com/group/rubyonrails-talk?hl=en. > > >-- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
Thanks! I don''t have any complicated logic hanging on the save code and only trivial validations. I''ll just use the connection.execute and that''s it. Amir -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.