I am a new convert to RoR and am loving every minute of it. Absolute
highest praise goes to everyone who has contributed.
An issue has come up when dealing with tables with large numbers of
rows and columns. Specifically, i have an issue in an
ActiveRecord::Migration, where after adding a new column, I would like
to iterate each row, read an attribute, process that attribute, then
update the new attribute/column with the result.
however, doing something like this:
Stuff.find_all { |r| r[''new_attr''] = process
r[''old_attr''] }
would kill the server returning so many records. I am aware of the
:select parameter to find() but it would still kill the server with ALL
records being returned in the array.
I see there is a :limit and :offset parameter to find(), which would
work. I figured i would ask if there are there any other workable
solutions to dealing with large data sets, before creating my own.
Thank you!
Jae
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk
-~----------~----~----~----~------~----~------~--~---
> I am a new convert to RoR and am loving every minute of it. Absolute > highest praise goes to everyone who has contributed. > > An issue has come up when dealing with tables with large numbers of > rows and columns. Specifically, i have an issue in an > ActiveRecord::Migration, where after adding a new column, I would like > to iterate each row, read an attribute, process that attribute, then > update the new attribute/column with the result. > > however, doing something like this: > Stuff.find_all { |r| r[''new_attr''] = process r[''old_attr''] } > > would kill the server returning so many records. I am aware of the > :select parameter to find() but it would still kill the server with ALL > records being returned in the array. > > I see there is a :limit and :offset parameter to find(), which would > work. I figured i would ask if there are there any other workable > solutions to dealing with large data sets, before creating my own.Just saw this on the rss feed yesterday which might work for you... Plugins - paginating_find - Agile Web Development http://www.agilewebdevelopment.com/plugins/paginating_find Got 15,842 records that you.d like to export to a file? Using the standard the Rails ActiveRecord::Base#find method will load all 15,842 into memory all at once and return them all in an array. If your app is running on a shared host, or if you.re keeping your app on a memory budget, this is a big problem for you. So you could load each record one by one, but that.ll kill your db server. Wouldn.t it be sweet if #find could return an enumerable that would load your records in batches of say 1,500 records? Well with my new nifty-jifty paginating_find plugin, it can. -philip --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk -~----------~----~----~----~------~----~------~--~---
Thanks Philip... i''m in nirvana. amazing. now i have to find out what plugins are. it''s amazing. a single line install, used it, tested it, it friggin works. amazing! Jae --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk -~----------~----~----~----~------~----~------~--~---
You may find the recently released paginating_find plugin helpful: http://agilewebdevelopment.com/plugins/paginating_find -- Building an e-commerce site with Rails? http://www.agilewebdevelopment.com/rails-ecommerce On Aug 29, 2006, at 11:00 AM, JChang wrote:> > I am a new convert to RoR and am loving every minute of it. Absolute > highest praise goes to everyone who has contributed. > > An issue has come up when dealing with tables with large numbers of > rows and columns. Specifically, i have an issue in an > ActiveRecord::Migration, where after adding a new column, I would like > to iterate each row, read an attribute, process that attribute, then > update the new attribute/column with the result. > > however, doing something like this: > Stuff.find_all { |r| r[''new_attr''] = process r[''old_attr''] } > > would kill the server returning so many records. I am aware of the > :select parameter to find() but it would still kill the server with > ALL > records being returned in the array. > > I see there is a :limit and :offset parameter to find(), which would > work. I figured i would ask if there are there any other workable > solutions to dealing with large data sets, before creating my own. > > Thank you! > > Jae > > > >--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk -~----------~----~----~----~------~----~------~--~---
Hi Jae. I''m glad to know the plugin is working well for you. Note that I recently updated the code so that it supports paging in a manual fashion, meaning that you can enumerate only over the current page and then step to the next page using the #next_page! method. This means that the plugin can be used really easily for both loading your pages and for displaying your pages. If you find yourself using the plugin in this fashion, I''d be interested to hear if it works well for you. -Alex http://cardboardrocket.com JChang wrote:> Thanks Philip... i''m in nirvana. amazing. now i have to find out what > plugins are. > > it''s amazing. a single line install, used it, tested it, it friggin > works. amazing! > > Jae-- Posted via http://www.ruby-forum.com/. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk -~----------~----~----~----~------~----~------~--~---