Hi All, I was wondering if anyone had run into a similar problem and had advice: I''m using ActiveRecord (without the rest of Rails) to manage data import for large collections of CSV data files into a database. The problem is that each CSV file comes from a different source, and each source uses completely different conventions for encoding just about everything (Country names, phone numbers, dates, null values, etc) Is there an easy way to sneak a step into the fixture loading process whereupon I have the chance to normalize the data in each row before it is dumped into the database? For example, I''d need to turn: John,Doe,us,4/23/1970,null,null,yes into: John,Doe,USA,1970-23-4,,1 I know I could just monkeypatch the Rails source, but I was hoping for a cleaner solution. Thanks, Ted [Singing the data integration blues] --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Edward wrote:> Hi All, > > I was wondering if anyone had run into a similar problem and had > advice: > > I''m using ActiveRecord (without the rest of Rails) to manage data > import for large collections of CSV data files into a database. The > problem is that each CSV file comes from a different source, and each > source uses completely different conventions for encoding just about > everything (Country names, phone numbers, dates, null values, etc) > > Is there an easy way to sneak a step into the fixture loading process > whereupon I have the chance to normalize the data in each row before > it is dumped into the database? > > For example, I''d need to turn: > > John,Doe,us,4/23/1970,null,null,yes > > into: > > John,Doe,USA,1970-23-4,,1 > > I know I could just monkeypatch the Rails source, but I was hoping for > a cleaner solution. > > Thanks, > Ted [Singing the data integration blues]If these are test fixtures why not have a separate script that massages the data beforehand? Otherwise you''ll need to do this "translation" everytime you run a test. -- Michael Wang --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---