Rails - Sep 2006 - Massive data insert during migration

Hello all,
I''ve got a site that will be used by people (hopefully) all over the
world. One of the requirements of registration is country and province of
residence. I have a list of all the countries in the world and all their
subnational entities. Currently, the two lists are divided between a pair of
comma-delimited text files. If this were something less dynamic and cool,
I''d probably import it into Excel or some equivalent and then do some
find-and-replace voodoo to create the SQL statements to enter it into the
database. What would you do if you were doing this in rails and wanted to make
sure this data was inserted when you run a migrate task?

Will

 		
---------------------------------
Yahoo! Messenger with Voice. Make PC-to-Phone Calls to the US (and 30+
countries) for 2¢/min or less.

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk
-~----------~----~----~----~------~----~------~--~---

I''m a rails novice, but I''m a sucker for the least pain
approach.

This is borderline - for an initial data load this small I would still
be tempted to do the Excel voodoo.

This information is generally static (apart from major world wars and
collapse of national governments), so the tables should be pretty
static.  If this particular table (or table set) is dynamic enough to
require a migration (i.e. columns will change), I would be tempted to
look at my designs again for a flaw in the data model.

The tool that I would use for an ongoing migration effort or something
really sizeable is not publicly available   :o(   but most RDBMS''s come
with fast unload and load utilities that can handle this sort of task,
provided the data structure does not change much.

Finally, I can actually mount comma delimited files as external tables
in the RDBMS that I prefer for rails projects, so my initial import
might be to look up how to mount the external table, then "select *
from oldtable into newtable", or "insert into newtable (col1, col1,
...) select (col1, col2, ...) from oldtable", depending on the exact
requirements of the new table.

I know that this isn''t exactly what you are looking for, but I hope it
helps.


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk
-~----------~----~----~----~------~----~------~--~---

I was thinking along the same lines, but after doing some research, it looks
like I could probably use the excel voodoo to generate some fixtures and load
the data that way. At least, I think that would work.

Can anyone confirm this?

johnson_d-j9pdmedNgrk@public.gmane.org wrote: 
I''m a rails novice, but I''m a sucker for the least pain
approach.

This is borderline - for an initial data load this small I would still
be tempted to do the Excel voodoo.

This information is generally static (apart from major world wars and
collapse of national governments), so the tables should be pretty
static.  If this particular table (or table set) is dynamic enough to
require a migration (i.e. columns will change), I would be tempted to
look at my designs again for a flaw in the data model.

The tool that I would use for an ongoing migration effort or something
really sizeable is not publicly available   :o(   but most RDBMS''s come
with fast unload and load utilities that can handle this sort of task,
provided the data structure does not change much.

Finally, I can actually mount comma delimited files as external tables
in the RDBMS that I prefer for rails projects, so my initial import
might be to look up how to mount the external table, then "select *
from oldtable into newtable", or "insert into newtable (col1, col1,
...) select (col1, col2, ...) from oldtable", depending on the exact
requirements of the new table.

I know that this isn''t exactly what you are looking for, but I hope it
helps.





 		
---------------------------------
 All-new Yahoo! Mail - Fire up a more powerful email and get things done faster.

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk
-~----------~----~----~----~------~----~------~--~---

I have certainly used Excel as a code generator in the past.


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk
-~----------~----~----~----~------~----~------~--~---

Will Gant wrote:
> Hello all,
> I''ve got a site that will be used by people (hopefully) all over
the
> world. One of the requirements of registration is country and province 
> of residence. I have a list of all the countries in the world and all 
> their subnational entities. Currently, the two lists are divided 
> between a pair of comma-delimited text files. If this were something 
> less dynamic and cool, I''d probably import it into Excel or some 
> equivalent and then do some find-and-replace voodoo to create the SQL 
> statements to enter it into the database. What would you do if you 
> were doing this in rails and wanted to make sure this data was 
> inserted when you run a migrate task?
>
> Will
You could convert it to YAML and then import it with your migration 
using the ''migration data dumper''

I wrote a post in this last month:
http://livsey.org/articles/2006/08/15/populating-the-database-with-migrations

The plugin:
http://blog.tammersaleh.com/articles/2006/08/15/migration-data-dumper-plugin

hth

--
Richard Livsey
http://livsey.org

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk
-~----------~----~----~----~------~----~------~--~---