Rails - Apr 2010 - Best practice around putting rails database info into git

This maybe too vague a question, but I wonder if there''s some wisdom
out
there for me to tap...

Sometimes when I get ready to commit, git says that db/schema.rb has
changed. This is weird, and I think it''s because I did a rake
db:migrate.

So my question is, what''s the best practice relating to putting bits of
the database (schema.rb and the databases themselves) into sourc
control. And what''s the best practice relating to running a rake
db:migrate after a git checkout of a new branch?

Anyone?

- Pito
-- 
Posted via http://www.ruby-forum.com/.

-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en.

1. I usually put my db/schema.rb to .gitignore. There''s some rake task
to
create your database not from migrations, but from schema.rb. But I
didn''t
see anybody alive doing this. So, you don''t really need this file in
your
repo.

2. You run your new migrations as soon as you get them. So, after pulling
new changes from remote branch



On Tue, Apr 20, 2010 at 9:25 PM, Pito Salas
<lists-fsXkhYbjdPsEEoCn2XhGlw@public.gmane.org> wrote:
> This maybe too vague a question, but I wonder if there''s some
wisdom out
> there for me to tap...
>
> Sometimes when I get ready to commit, git says that db/schema.rb has
> changed. This is weird, and I think it''s because I did a rake
> db:migrate.
>
> So my question is, what''s the best practice relating to putting
bits of
> the database (schema.rb and the databases themselves) into sourc
> control. And what''s the best practice relating to running a rake
> db:migrate after a git checkout of a new branch?
>
> Anyone?
>
> - Pito
> --
> Posted via http://www.ruby-forum.com/.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Ruby on Rails: Talk" group.
> To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To unsubscribe from this group, send email to
>
rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org<rubyonrails-talk%2Bunsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org>
> .
> For more options, visit this group at
> http://groups.google.com/group/rubyonrails-talk?hl=en.
>
>
-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en.

Vladimir Rybas wrote:
> 1. I usually put my db/schema.rb to .gitignore. There''s some rake
task
> to
> create your database not from migrations, but from schema.rb. But I 
> didn''t
> see anybody alive doing this. So, you don''t really need this file
in
> your
> repo.
> 
> 2. You run your new migrations as soon as you get them. So, after 
> pulling
> new changes from remote branch
Thanks!!

-- Pito

-- 
Posted via http://www.ruby-forum.com/.

-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en.

Pito Salas wrote:
> This maybe too vague a question, but I wonder if there''s some
wisdom out
> there for me to tap...
> 
> Sometimes when I get ready to commit, git says that db/schema.rb has
> changed. This is weird, and I think it''s because I did a rake
> db:migrate.
> 
> So my question is, what''s the best practice relating to putting
bits of
> the database (schema.rb and the databases themselves) into sourc
> control. And what''s the best practice relating to running a rake
> db:migrate after a git checkout of a new branch?
Beyond the issues with db/schema.rb, I think it''s also good practice to
not version control config/database.yml. You may be doing this already.

The reason for this is that database configurations for other member of 
the team may be different, and you may not want to expose the production 
database configuration to then entire team.

I do the following:

1. Rename a basic config/database.yml file to 
config/example_database.yml.
2. Add config/example_database.yml to Git
3. Ignore config/database.yml in .gitignore
4. Commit these changes
5. Copy config/example_database.yml to config/database.yml

When other team members clone the repository they will also need to copy 
the example_database.yml file and configure it for their own system. 
This allow more freedom to other developers on the team to setup their 
system the way they see fit.

For example: I may prefer to use the MySQL root user with no password 
set on my development database, while other may prefer (or need) their 
development database protected by using a different user with their own 
password. Or, maybe I might prefer using SQLite in development while 
others on the team prefer developing using MySQL.

Keeping this configuration out of the versioning system makes these 
kinds of preference easier to manage. This technique also helps in cases 
where not all team member are allowed to know the production database''s
password.
-- 
Posted via http://www.ruby-forum.com/.

-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en.

Vladimir Rybas wrote:
> 1. I usually put my db/schema.rb to .gitignore. There''s some rake
task
> to
> create your database not from migrations, but from schema.rb. But I 
> didn''t
> see anybody alive doing this. 
I do it.
> So, you don''t really need this file in 
> your
> repo.
Completely wrong.  You should always create your new installations'' 
databases with rake db:schema:load , *never* by running migrations from 
zero.  Migrations are only for changing *existing* installations, not 
for creating new ones from scratch.

Therefore, you need db/schema.rb in version control.  There''s even a 
comment in that file that says as much.

config/database.yml shouldn''t be in version control because it contains
passwords, but I like to put a sample version of that file (without 
passwords) in the repository as an example of how it should be set up.
> 
> 2. You run your new migrations as soon as you get them. So, after 
> pulling
> new changes from remote branch
Right.

Best,
--
Marnen Laibow-Koser
http://www.marnen.org
marnen-sbuyVjPbboAdnm+yROfE0A@public.gmane.org

-- 
Posted via http://www.ruby-forum.com/.

-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en.

Pito Salas wrote:
> This maybe too vague a question, but I wonder if there''s some
wisdom out
> there for me to tap...
> 
> Sometimes when I get ready to commit, git says that db/schema.rb has
> changed. This is weird, and I think it''s because I did a rake
> db:migrate.
Yes, that would probably be why.  Why do you think this is "weird"? 
It''s normal behavior -- schema.rb contains what Rails thinks your 
database schema is.
> 
> So my question is, what''s the best practice relating to putting
bits of
> the database (schema.rb and the databases themselves)
schema.rb is not a "bit of the database".
> into sourc
> control. And what''s the best practice relating to running a rake
> db:migrate after a git checkout of a new branch?
To summarize my earlier posts: schema.rb definitely goes into version 
control.  I don''t bother putting SQLite databases themselves in the 
repository, though (not that I use SQLite much).
> 
> Anyone?
> 
> - Pito
Best,
--
Marnen Laibow-Koser
http://www.marnen.org
marnen-sbuyVjPbboAdnm+yROfE0A@public.gmane.org
-- 
Posted via http://www.ruby-forum.com/.

-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en.

Pito,

I''m going to leave the entire db version control question for someone
else, but to answer your question about the schema.rb, if you run a
migration, it will change the structure of the database, which is what
schema.rb keeps track of. Rails will rewrite that file every for every
migration ran to reflect the current structure of the db ( structure
as in tables, column names, indexes, etc... Not data the records
stored in the db ). This file should be checked into git, and like any
file that changes under version control, git will let you know about
the changes and track them.

As your app grows, your migration files will be more prone to hving
problems and shouldn''t be used to create / setup the database. The
schema file is much better at recreating the db structure when
starting fresh ( like if you have to scale up your app and create a
new db on another server ). There are rake tasks for this, such as
rake db:reset ( caution, that one will wipe out all data in db), so
read up on them and they can save you many headaches as you have to
move your databases around.

-Matt

On Apr 20, 9:25 am, Pito Salas
<li...-fsXkhYbjdPsEEoCn2XhGlw@public.gmane.org>
wrote:
> This maybe too vague a question, but I wonder if there''s some
wisdom out
> there for me to tap...
>
> Sometimes when I get ready to commit, git says that db/schema.rb has
> changed. This is weird, and I think it''s because I did a rake
> db:migrate.
>
> So my question is, what''s the best practice relating to putting
bits of
> the database (schema.rb and the databases themselves) into sourc
> control. And what''s the best practice relating to running a rake
> db:migrate after a git checkout of a new branch?
>
> Anyone?
>
> - Pito
> --
> Posted viahttp://www.ruby-forum.com/.
>
> --
> You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
> To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe@googlegroups.com.
> For more options, visit this group
athttp://groups.google.com/group/rubyonrails-talk?hl=en.
-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en.

Matt,

Thanks for a great summary; very helpful, and thanks to the rest of the 
list for always providing insightful comments and solutions to problems. 
Great!!

- Pito

Matt wrote:
> Pito,
> 
> I''m going to leave the entire db version control question for
someone
> else, but to answer your question about the schema.rb, if you run a
> migration, it will change the structure of the database, which is what
> schema.rb keeps track of. Rails will rewrite that file every for every
> migration ran to reflect the current structure of the db ( structure
> as in tables, column names, indexes, etc... Not data the records
> stored in the db ). This file should be checked into git, and like any
> file that changes under version control, git will let you know about
> the changes and track them.
> 
> As your app grows, your migration files will be more prone to hving
> problems and shouldn''t be used to create / setup the database. The
> schema file is much better at recreating the db structure when
> starting fresh ( like if you have to scale up your app and create a
> new db on another server ). There are rake tasks for this, such as
> rake db:reset ( caution, that one will wipe out all data in db), so
> read up on them and they can save you many headaches as you have to
> move your databases around.
> 
> -Matt
-- 
Posted via http://www.ruby-forum.com/.

-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en.

On Apr 21, 2010, at 3:12 AM, Matt wrote:
>
> On Apr 20, 9:25 am, Pito Salas
<li...-fsXkhYbjdPsEEoCn2XhGlw@public.gmane.org> wrote:
>> This maybe too vague a question, but I wonder if there''s some
>> wisdom out
>> there for me to tap...
>>
>> Sometimes when I get ready to commit, git says that db/schema.rb has
>> changed. This is weird, and I think it''s because I did a rake
>> db:migrate.
>>
>> So my question is, what''s the best practice relating to
putting
>> bits of
>> the database (schema.rb and the databases themselves) into sourc
>> control. And what''s the best practice relating to running a
rake
>> db:migrate after a git checkout of a new branch?
>>
>> Anyone?
>>
>> - Pito
>
> Pito,
>
> I''m going to leave the entire db version control question for
someone
> else, but to answer your question about the schema.rb, if you run a
> migration, it will change the structure of the database, which is what
> schema.rb keeps track of. Rails will rewrite that file every for every
> migration ran to reflect the current structure of the db ( structure
> as in tables, column names, indexes, etc... Not data the records
> stored in the db ). This file should be checked into git, and like any
> file that changes under version control, git will let you know about
> the changes and track them.
>
> As your app grows, your migration files will be more prone to hving
> problems and shouldn''t be used to create / setup the database. The
> schema file is much better at recreating the db structure when
> starting fresh ( like if you have to scale up your app and create a
> new db on another server ). There are rake tasks for this, such as
> rake db:reset ( caution, that one will wipe out all data in db), so
> read up on them and they can save you many headaches as you have to
> move your databases around.
>
> -Matt
Matt, Pito, Marnen, and anyone else,

1. The opinion on whether db/schema.rb goes into the source repository  
has changed over time.  It is an opinion. (I''ll give you some of mine.)

2. When properly written, migration files are not "prone to having  
problems" and certainly do not have to get worse over time. The advise  
here is to define at least a minimal version of your AR model class  
nested within the migration class if you need to do *any* manipulation  
at the model level.

3. I think that db/schema.rb does *NOT* belong in the repository.   
This is particularly true in a multi-developer environment where  
migrations are being initially created on different branches. The  
database itself isn''t going into the repository after all. If you need
the file, run a rake db:schema:dump or just run migrations. If you  
think about why the migration numbering (file naming) was changed from  
sequence number to timestamp, you''ll realize that the practice of such
"interleaved" migrations was a much bigger pain-point than what to do
about db/schema.rb.

4. Unless you''re initializing (inserting) data via migration, running  
all the migrations is really not much different than doing a  
db:schema:load because all the migrations are operating on empty  
tables. (Besides, if you have to "scale up your app", you probably  
aren''t adding a new empty database, but creating a master-slave or  
sharding for performance.)

-Rob

Rob Biedenharn		http://agileconsultingllc.com
Rob-xa9cJyRlE0mWcWVYNo9pwxS2lgjeYSpx@public.gmane.org



-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en.

Rob Biedenharn wrote:
[...]
> Matt, Pito, Marnen, and anyone else,
> 
> 1. The opinion on whether db/schema.rb goes into the source repository
> has changed over time.
No.  I''ve used Rails since 1.2.6.  Every version has put a comment in 
the schema.rb file that recommends putting it into version control.
>  It is an opinion. (I''ll give you some of mine.)
Opinion is not an excuse for advocating dangerous practices.
> 
> 2. When properly written, migration files are not "prone to having
> problems" and certainly do not have to get worse over time. The advise
> here is to define at least a minimal version of your AR model class
> nested within the migration class if you need to do *any* manipulation
> at the model level.
But you should never be running old migrations in the first place.  If 
you need version 1000 of the schema for a new installation, then don''t 
start at zero and run 1000 migrations -- just do rake db:schema:load and 
have done with it.  This is the core team''s recommendation, and I think
it''s a good one.
> 
> 3. I think that db/schema.rb does *NOT* belong in the repository.
Why not?  Without it, you can''t load the schema with Rake.
> This is particularly true in a multi-developer environment where
> migrations are being initially created on different branches.
I don''t see how this makes a difference.  Your VCS should be able to 
merge the schema.rb files.  If not, get a better VCS.
> The
> database itself isn''t going into the repository after all. 
That''s a red herring.
> If you need
> the file, run a rake db:schema:dump or just run migrations.
No!  You need the file so that rake db:schema:load is possible.  You''ve
got it completely backwards.
>  If you
> think about why the migration numbering (file naming) was changed from
> sequence number to timestamp, you''ll realize that the practice of
such
> "interleaved" migrations was a much bigger pain-point than what
to do
> about db/schema.rb.
I don''t really understand what you''re getting at here.
> 
> 4. Unless you''re initializing (inserting) data via migration,
running
> all the migrations is really not much different than doing a
> db:schema:load because all the migrations are operating on empty
> tables.
How can you say this with a straight face?  There is no reason *at all* 
to run lots of migrations rather than doing a simple schema load.
> (Besides, if you have to "scale up your app", you probably
> aren''t adding a new empty database, but creating a master-slave or
> sharding for performance.)
Another red herring.

Rob, I know you know a lot about the Rails framework, but your advice 
here will make dealing with databases far more difficult than it needs 
to be.
> 
> -Rob
> 
> Rob Biedenharn    http://agileconsultingllc.com
> Rob-xa9cJyRlE0mWcWVYNo9pwxS2lgjeYSpx@public.gmane.org
Best,
-- 
Marnen Laibow-Koser
http://www.marnen.org
marnen-sbuyVjPbboAdnm+yROfE0A@public.gmane.org
-- 
Posted via http://www.ruby-forum.com/.

-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en.

On Apr 22, 2010, at 9:06 AM, Marnen Laibow-Koser wrote:
> Rob Biedenharn wrote:
> [...]
>> Matt, Pito, Marnen, and anyone else,
>>
>> 1. The opinion on whether db/schema.rb goes into the source  
>> repository
>> has changed over time.
>
> No.  I''ve used Rails since 1.2.6.  Every version has put a comment
in
> the schema.rb file that recommends putting it into version control.
I''ve been using Rails since 0.13 and I''ll restate that this
opinion
has changed.
I can''t find it at the moment, but there WAS a version that  
specifically said in
that comment to NOT put the file into source control. At least in  
1.2.2, there
is no comment either way.
>
>> It is an opinion. (I''ll give you some of mine.)
>
> Opinion is not an excuse for advocating dangerous practices.
"Dangerous"? Well, that''s certainly an opinion. ;-)

>
>>
>> 2. When properly written, migration files are not "prone to having
>> problems" and certainly do not have to get worse over time. The  
>> advise
>> here is to define at least a minimal version of your AR model class
>> nested within the migration class if you need to do *any*  
>> manipulation
>> at the model level.
>
> But you should never be running old migrations in the first place.  If
> you need version 1000 of the schema for a new installation, then
don''t
> start at zero and run 1000 migrations -- just do rake db:schema:load  
> and
> have done with it.  This is the core team''s recommendation, and I
> think
> it''s a good one.
I have helped other developers who have created several migrations in  
development, which were applied approximately when created, that were  
subsequently unable to run when deployed to production.  The very same  
recommendations that guard against this kind of problem will make it  
possible to run those 1000 migrations (or any subsequence) without  
problem.

>
>>
>> 3. I think that db/schema.rb does *NOT* belong in the repository.
>
> Why not?  Without it, you can''t load the schema with Rake.
>
>> This is particularly true in a multi-developer environment where
>> migrations are being initially created on different branches.
>
> I don''t see how this makes a difference.  Your VCS should be able
to
> merge the schema.rb files.  If not, get a better VCS.
Yikes! No! The database itself holds the official version of the  
schema. If I merge changes from a master branch into my development  
branch, I will run any new migrations, but I certainly don''t want some
merge tool to give me a new schema.rb.  Depending on the actual  
content of the migrations on different branches and the order in which  
they are run, the *actual* schema might be slightly different due to  
the rules for where new columns are placed on a table.


>
>> The
>> database itself isn''t going into the repository after all.
>
> That''s a red herring.
>
>> If you need
>> the file, run a rake db:schema:dump or just run migrations.
>
> No!  You need the file so that rake db:schema:load is possible.   
> You''ve
> got it completely backwards.
You assume that I need to run db:schema:load, which *I* don''t.

>
>> If you
>> think about why the migration numbering (file naming) was changed  
>> from
>> sequence number to timestamp, you''ll realize that the practice
of
>> such
>> "interleaved" migrations was a much bigger pain-point than
what to do
>> about db/schema.rb.
>
> I don''t really understand what you''re getting at here.
When migrations were sequentially numbered, two developers on separate  
branches might both create migration 005 for different purposes. This  
was a problem.  The chance that two developers both create migration  
20100422105524 is acceptable small. Of course, they will probably be  
executed in a different order and if they both add a column to the  
same table, those columns will likewise be in a different order. If db/ 
schema.rb is in the repository, then lots of commits will have  
effectively meaningless changes and unless the current HEAD has *my*  
version is it *not* going to truly represent what''s in my database  
schema.
>
>>
>> 4. Unless you''re initializing (inserting) data via migration,
running
>> all the migrations is really not much different than doing a
>> db:schema:load because all the migrations are operating on empty
>> tables.
>
> How can you say this with a straight face?  There is no reason *at  
> all*
> to run lots of migrations rather than doing a simple schema load.
The reason is deploying to an existing production database.  You can''t
do a schema load.  The proper way to apply the "new" migrations (which
might be kinda old if the last production deploy wasn''t so recent) is  
a db:migrate. As I''ve stated, you can get into trouble with mismatches
between migrations (which don''t change after being created and  
*shouldn''t* if created properly) and models (which obviously change  
over time).

I have projects with 210 and 204 migrations as well as many with  
fewer. Some of the migrations deal with rather nasty data  
manipulations to maintain data relationships when the associations are  
flipped around. It''s not something that I would recommend, but the  
definition of "reasonable" can change dramatically when a client  
shifts the way he thinks about the data and its evolution.

As an experiment, I set up a new environment for the 204 migration  
project and ran the migrations from scratch. It takes about 8  
minutes.  There is about 1 minute of startup time, there are several  
migrations that load some data including one that takes a bit over 3  
minutes to put a few tens of thousands of research datapoints into a  
set of tables.  I''m OK with that amount of time for something as  
significant as creating a new environment.

>
>> (Besides, if you have to "scale up your app", you probably
>> aren''t adding a new empty database, but creating a
master-slave or
>> sharding for performance.)
>
> Another red herring.
Well, if you can "scale up your app" by starting from an empty schema,
go ahead. Perhaps initializing the shards, but then I''d start with the
db/schema.rb from production, not something from the repository which  
almost certainly reflects a development environment however close to  
production that might be.
>
> Rob, I know you know a lot about the Rails framework, but your advice
> here will make dealing with databases far more difficult than it needs
> to be.
If I help someone who recalls some of these nuggets of my wisdom and  
experience at a time where migrations give them trouble, then I will  
have made a positive difference.

However, it is that same experience that has led me to the conclusion  
that keeping db/schema.rb in the source repository is wrong.  It is  
derived data and I would no more put it into the repository than I  
would have someone put their object files compiled from C or their  
class files compiled from Java in there.
>
>>
>> -Rob
>>
>> Rob Biedenharn    http://agileconsultingllc.com
>> Rob-xa9cJyRlE0mWcWVYNo9pwxS2lgjeYSpx@public.gmane.org
>
> Best,
> -- 
> Marnen Laibow-Koser
> http://www.marnen.org
> marnen-sbuyVjPbboAdnm+yROfE0A@public.gmane.org
> -- 
I''m sure we''ll meet again! Take care,

-Rob

Rob Biedenharn		http://agileconsultingllc.com
Rob-xa9cJyRlE0mWcWVYNo9pwxS2lgjeYSpx@public.gmane.org



-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en.

On Apr 22, 4:04 pm, Rob Biedenharn
<R...-xa9cJyRlE0mWcWVYNo9pwxS2lgjeYSpx@public.gmane.org>
wrote:
>
> However, it is that same experience that has led me to the conclusion  
> that keeping db/schema.rb in the source repository is wrong.  It is  
> derived data and I would no more put it into the repository than I  
> would have someone put their object files compiled from C or their  
> class files compiled from Java in there.
>
I think there is a difference between an app like mephisto or similar
where many people are going to be deploying instances all over the
place and a private app for which there is only ever going to be one
deploy. With the latter, the database is the definition of the schema
and you don''t care about people deploying the app from scratch because
no one will

Fred

-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en.

Rob Biedenharn wrote:
> On Apr 22, 2010, at 9:06 AM, Marnen Laibow-Koser wrote:
> 
>> Rob Biedenharn wrote:
>> [...]
>>> Matt, Pito, Marnen, and anyone else,
>>>
>>> 1. The opinion on whether db/schema.rb goes into the source  
>>> repository
>>> has changed over time.
>>
>> No.  I''ve used Rails since 1.2.6.  Every version has put a
comment in
>> the schema.rb file that recommends putting it into version control.
> 
> I''ve been using Rails since 0.13 and I''ll restate that
this opinion
> has changed.
> I can''t find it at the moment, but there WAS a version that
> specifically said in
> that comment to NOT put the file into source control. At least in
> 1.2.2, there
> is no comment either way.
I''ll get out my old versions of Rails and look.  However, it''s
pretty
much irrelevant, because look what current versions say:

# Note that this schema.rb definition is the authoritative source for 
your database schema. If you need
# to create the application database on another system, you should be 
using db:schema:load, not running
# all the migrations from scratch. The latter is a flawed and 
unsustainable approach (the more migrations
# you''ll amass, the slower it''ll run and the greater
likelihood for
issues).
#
# It''s strongly recommended to check this file into your version
control
system.

That''s pretty definitive.


[...]
>> But you should never be running old migrations in the first place.  If
>> you need version 1000 of the schema for a new installation, then
don''t
>> start at zero and run 1000 migrations -- just do rake db:schema:load  
>> and
>> have done with it.  This is the core team''s recommendation,
and I
>> think
>> it''s a good one.
> 
> I have helped other developers who have created several migrations in
> development, which were applied approximately when created, that were
> subsequently unable to run when deployed to production.  
And why was this so?
> The very same
> recommendations that guard against this kind of problem will make it
> possible to run those 1000 migrations (or any subsequence) without
> problem.
No.  There is a difference between running a days-old migration and a 
years-old migration.
> 
> 
>> merge the schema.rb files.  If not, get a better VCS.
> Yikes! No! The database itself holds the official version of the
> schema. If I merge changes from a master branch into my development
> branch, I will run any new migrations, but I certainly don''t want
some
> merge tool to give me a new schema.rb.  
For *changing* the schema on an existing installation, I agree with you 
-- run the migrations.  For a new installation, use rake db:schema:load. 
That implies that you need schema.rb in the VCS.  How much more clearly 
can I say this?
> Depending on the actual
> content of the migrations on different branches and the order in which
> they are run, the *actual* schema might be slightly different due to
> the rules for where new columns are placed on a table.
The order of columns on a DB table is immaterial.  You should be able to 
run rake db:schema:load for a new installation and immediately get a 
usable DB.
> 
> 
> 
>> You''ve
>> got it completely backwards.
> 
> You assume that I need to run db:schema:load, which *I* don''t.
Why do you believe you don''t?  Because you don''t understand
what it''s
for?  Because you never do new installations?

I''ll submit that you *do* need to run db:schema:load, and you just
don''t
know it because you don''t understand what it''s good for.
> 
> 
>>
>>> If you
>>> think about why the migration numbering (file naming) was changed  
>>> from
>>> sequence number to timestamp, you''ll realize that the
practice of
>>> such
>>> "interleaved" migrations was a much bigger pain-point
than what to do
>>> about db/schema.rb.
>>
>> I don''t really understand what you''re getting at
here.
> 
> When migrations were sequentially numbered, two developers on separate
> branches might both create migration 005 for different purposes. This
> was a problem.  The chance that two developers both create migration
> 20100422105524 is acceptable small. 
OK, now I see what you mean.  Yes.
> Of course, they will probably be
> executed in a different order and if they both add a column to the
> same table, those columns will likewise be in a different order. 
So what?  Column order is immaterial.
> If db/
> schema.rb is in the repository, then lots of commits will have
> effectively meaningless changes 
How do you figure that?
> and unless the current HEAD has *my*
> version is it *not* going to truly represent what''s in my database
> schema.
Sure it is -- modulo immaterial things like column order.
> 
>>
>>>
>>> 4. Unless you''re initializing (inserting) data via
migration, running
>>> all the migrations is really not much different than doing a
>>> db:schema:load because all the migrations are operating on empty
>>> tables.
>>
>> How can you say this with a straight face?  There is no reason *at  
>> all*
>> to run lots of migrations rather than doing a simple schema load.
> 
> The reason is deploying to an existing production database.  You
can''t
> do a schema load.  
Right.  That''s what migrations are for.  That''s the *only*
thing
migrations are for -- changing an existing database.
> The proper way to apply the "new" migrations (which
> might be kinda old if the last production deploy wasn''t so recent)
Then you''ve got bigger problems.  If you can''t do lots of
little
deploys, then you will wind up in integration hell anyway, and 
migrations will be the least of your worries.
> is
> a db:migrate. As I''ve stated, you can get into trouble with
mismatches
> between migrations (which don''t change after being created and
> *shouldn''t* if created properly) and models (which obviously
change
> over time).
If you''re deploying frequently enough, there will be no issue.
> 
> I have projects with 210 and 204 migrations as well as many with
> fewer. Some of the migrations deal with rather nasty data
> manipulations to maintain data relationships when the associations are
> flipped around. It''s not something that I would recommend, but the
> definition of "reasonable" can change dramatically when a client
> shifts the way he thinks about the data and its evolution.
> 
> As an experiment, I set up a new environment for the 204 migration
> project and ran the migrations from scratch. It takes about 8
> minutes.  
That''s a long time, but that''s the least of the problems.
> There is about 1 minute of startup time, there are several
> migrations that load some data including one that takes a bit over 3
> minutes to put a few tens of thousands of research datapoints into a
> set of tables.  
Wow, it just gets worse. :)  Seed data never, ever belongs in migrations 
-- partly because it makes schema loading impossible, and partly because 
migrations are only about the schema, not the data.  Use seed-fu or 
Rails 2.3''s built-in seeding.
> I''m OK with that amount of time for something as
> significant as creating a new environment.
Yes, it''s a one-time task.  But it''s still wrong. :)
> 
> 
>>
>>> (Besides, if you have to "scale up your app", you
probably
>>> aren''t adding a new empty database, but creating a
master-slave or
>>> sharding for performance.)
>>
>> Another red herring.
> 
> Well, if you can "scale up your app" by starting from an empty
schema,
> go ahead. Perhaps initializing the shards, but then I''d start with
the
> db/schema.rb from production, not something from the repository which
> almost certainly reflects a development environment however close to
> production that might be.
Still a red herring.  I want new installations to be easy to create -- 
not for scalability, which would certainly use DB replication, but for 
people actually creating a new, independent instance of the app -- say, 
a test environment, or simply someone else installing a Rails app
that''s
meant for external use.
> 
>>
>> Rob, I know you know a lot about the Rails framework, but your advice
>> here will make dealing with databases far more difficult than it needs
>> to be.
> 
> If I help someone who recalls some of these nuggets of my wisdom and
> experience at a time where migrations give them trouble, then I will
> have made a positive difference.
> 
> However, it is that same experience that has led me to the conclusion
> that keeping db/schema.rb in the source repository is wrong.  It is
> derived data and I would no more put it into the repository than I
> would have someone put their object files compiled from C or their
> class files compiled from Java in there.
schema.rb is certainly derived data to a point. However, it is also the 
only reliable source of information about what the DB schema should look 
like at a given point in history.  I probably would not have thought to 
put it into version control had I not seen that note from the core team, 
but having done so and thought about it, I believe that it must go into 
version control.  It contains information that cannot be reliably 
derived except by going through the process that it was designed to 
circumvent.

To continue the compilation analogy: you might not put your binaries 
into the repository with the source code -- but you''d surely want to 
make them available for people who didn''t care to build from source.

There are many good reasons to have schema.rb in version control.  There 
are few if any to do otherwise.

Best,
--
Marnen Laibow-Koser
http://www.marnen.org
marnen-sbuyVjPbboAdnm+yROfE0A@public.gmane.org
-- 
Posted via http://www.ruby-forum.com/.

-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en.

Frederick Cheung wrote:
> On Apr 22, 4:04�pm, Rob Biedenharn
<R...-xa9cJyRlE0mWcWVYNo9pwxS2lgjeYSpx@public.gmane.org>
> wrote:
> 
>>
>> However, it is that same experience that has led me to the conclusion �
>> that keeping db/schema.rb in the source repository is wrong. �It is �
>> derived data and I would no more put it into the repository than I �
>> would have someone put their object files compiled from C or their �
>> class files compiled from Java in there.
>>
> 
> I think there is a difference between an app like mephisto or similar
> where many people are going to be deploying instances all over the
> place and a private app for which there is only ever going to be one
> deploy. 
I disagree.  The severity is different, but the issues are the same.
> With the latter, the database is the definition of the schema
> and you don''t care about people deploying the app from scratch
because
> no one will
Two words: staging server.
> 
> Fred
Best,
--
Marnen Laibow-Koser
http://www.marnen.org
marnen-sbuyVjPbboAdnm+yROfE0A@public.gmane.org
-- 
Posted via http://www.ruby-forum.com/.

-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en.

On Apr 22, 2010, at 4:23 PM, Frederick Cheung wrote:
> On Apr 22, 4:04 pm, Rob Biedenharn
<R...-xa9cJyRlE0mWcWVYNo9pwxS2lgjeYSpx@public.gmane.org>
> wrote:
>
>> However, it is that same experience that has led me to the conclusion
>> that keeping db/schema.rb in the source repository is wrong.  It is
>> derived data and I would no more put it into the repository than I
>> would have someone put their object files compiled from C or their
>> class files compiled from Java in there.
>>
>
> I think there is a difference between an app like mephisto or similar
> where many people are going to be deploying instances all over the
> place and a private app for which there is only ever going to be one
> deploy. With the latter, the database is the definition of the schema
> and you don''t care about people deploying the app from scratch
because
> no one will
>
> Fred

"only ever going to be one deploy" ??

When does the app work right the very first time and never has to be  
changed?

Your second sentence sounds like you''ve come around to my point:
"the
database is the definition of the schema"
Something goes into the repository in order to track its changes over  
time, share it with other developers, and hold the definitive copy to  
be deployed. Things that *don''t* go into the repository are things  
that don''t need to be tracked (e.g., log files), aren''t or
shouldn''t
be shared (e.g., passwords and private config like database.yml for  
production), or aren''t the definitive copy (e.g., like db/schema.rb  
unless you''re putting it *back* into the repo from the production  
environment).

-Rob

Rob Biedenharn		http://agileconsultingllc.com
Rob-xa9cJyRlE0mWcWVYNo9pwxS2lgjeYSpx@public.gmane.org



-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en.

On Apr 22, 9:55 pm, Rob Biedenharn
<R...-xa9cJyRlE0mWcWVYNo9pwxS2lgjeYSpx@public.gmane.org>
wrote:
> On Apr 22, 2010, at 4:23 PM, Frederick Cheung wrote:
>
> > I think there is a difference between an app like mephisto or similar
> > where many people are going to be deploying instances all over the
> > place and a private app for which there is only ever going to be one
> > deploy. With the latter, the database is the definition of the schema
> > and you don''t care about people deploying the app from
scratch because
> > no one will
>
> > Fred
>
> "only ever going to be one deploy" ??
>
> When does the app work right the very first time and never has to be  
> changed?
Sorry that wasn''t very clear. a better phrasing might be "only one
deployment" (which you of course keep working on and updating as
needed)

Fred

-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en.