Hi, I''m trying to run a set of migration which brings data from a legacy project. The legacy project is on production and have lots of data. So, during the time the migration keep allocating and allocating memory until it grabs it all. I''m doing pagination to bring the data step by step. Beside, what is very wear is that the memory never goes down between migration, for example, I do: rake db:migrate == 2 InitialMoveLegacyUsersToUsers: migrating ================================== 2 InitialMoveLegacyUsersToUsers: migrated (83.3700s) ====================== == 3 AddPasswordResetCodeToUsers: migrating ==================================-- add_column(:users, :password_reset_code, :string) -> 0.0877s == 3 AddPasswordResetCodeToUsers: migrated (0.0883s) ========================= == 4 AddOpenIdAuthenticationTables: migrating ================================-- create_table(:open_id_authentication_associations, {:force=>true}) -> 0.0054s -- create_table(:open_id_authentication_nonces, {:force=>true}) -> 0.0044s == 4 AddOpenIdAuthenticationTables: migrated (0.0105s) ======================= And watching the behavior of the memory between migrations it never free memory, always keep growing. So, it seems to me like a memory leak, but dunno if I''m doing something wrong. Is there a way to call the garbage collector to free the memory between migrations?? I''ll appreciate your help. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Rafael Chacon wrote:> > rake db:migrate > > == 2 InitialMoveLegacyUsersToUsers: migrating > ================================> == 2 InitialMoveLegacyUsersToUsers: migrated (83.3700s) > ======================> > == 3 AddPasswordResetCodeToUsers: migrating > ==================================> -- add_column(:users, :password_reset_code, :string) > -> 0.0877s > == 3 AddPasswordResetCodeToUsers: migrated (0.0883s) > =========================> > == 4 AddOpenIdAuthenticationTables: migrating > ================================> -- create_table(:open_id_authentication_associations, {:force=>true}) > -> 0.0054s > -- create_table(:open_id_authentication_nonces, {:force=>true}) > -> 0.0044s > == 4 AddOpenIdAuthenticationTables: migrated (0.0105s) > ======================= > And watching the behavior of the memory between migrations it never > free memory, always keep growing.You probably need to show me what your migrations look like. You must be loading a load of database rows into ruby objects? Maybe you''re setting them to a variable that remains in scope? Don''t know. If you do rake db:migrate VERSION=2 and then seperately do rake db:migrate VERSION=4 does it isolate the problem to the first migration? -- Posted via http://www.ruby-forum.com/. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
The most conlficting migration looks like this: class MoveLegacySugarEntries < ActiveRecord::Migration class Entries < ActiveRecord::Base;end class Entry::SugarEntry < Entry;end def self.up Legacy::SugarReading.each do |le| u = User.find_by_login(le.user.login) unless u== nil TzTime.zone = u.tz Entry::SugarEntry.create!( :registered_at => DateTime.parse("#{le.date} #{le.time.hour}:#{le.time.min}"), :comment => le.comment, :value => le.reading, :user => u) end end end Where the active record each is implemented like this: class <<ActiveRecord::Base def each(limit=1000) rows = find(:all, :conditions => ["id > ?", 0], :limit => limit) while rows.any? rows.each { |record| yield record } rows = find(:all, :conditions => ["id > ?", rows.last.id], :limit => limit) end self end end So, is getting 1000 rows at a time... Enough to don''t exploit the memory. Which variable might not be getting out of scope?? Anyway, how would you think possible to get garbage memory from previous migrations?? For example, if I ran my migration from 0 when I got to this migration I already have 1.5 gigas memory occupied, isn''t kind of wear??? Thanks for your help On Jun 28, 3:37 am, Matthew Rudy Jacobs <rails-mailing-l...@andreas- s.net> wrote:> Rafael Chacon wrote: > > > rake db:migrate > > > == 2 InitialMoveLegacyUsersToUsers: migrating > > ================================> > == 2 InitialMoveLegacyUsersToUsers: migrated (83.3700s) > > ======================> > > == 3 AddPasswordResetCodeToUsers: migrating > > ==================================> > -- add_column(:users, :password_reset_code, :string) > > -> 0.0877s > > == 3 AddPasswordResetCodeToUsers: migrated (0.0883s) > > =========================> > > == 4 AddOpenIdAuthenticationTables: migrating > > ================================> > -- create_table(:open_id_authentication_associations, {:force=>true}) > > -> 0.0054s > > -- create_table(:open_id_authentication_nonces, {:force=>true}) > > -> 0.0044s > > == 4 AddOpenIdAuthenticationTables: migrated (0.0105s) > > =======================> > And watching the behavior of the memory between migrations it never > > free memory, always keep growing. > > You probably need to show me what your migrations look like. > You must be loading a load of database rows into ruby objects? > > Maybe you''re setting them to a variable that remains in scope? > > Don''t know. > > If you do rake db:migrate VERSION=2 > and then seperately do rake db:migrate VERSION=4 > > does it isolate the problem to the first migration? > -- > Posted viahttp://www.ruby-forum.com/.--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---