Rails - Apr 2006 - DevTower Beta3 Release!

DevTower Beta3! (0.3)

I''m proud to announce some major changes and improvements to DevTower
in
this beta release! First and foremost, DevTower is now a full plugin. Also,
DevTower works (only) with Rails 1.1.


==== ABOUT ====
DevTower used to synchronize the development of Ruby On
Rails applications with developers working simultaniously
on multiple systems with Subversion. DevTower is built
around the philosophy of responsible group-development
and enforces certain development practices which you may
or may not like. However, it is my belief that all are good
practices for any collaborative development system.

It is comprised of a few extensions to ActiveRecord::Base
(via a modified ar_fixtures) and several Rake tasks to speed
group development.

This was one of those things that was made for use in-house.
Many of you probably have scripts around that do similar things
to synchronize your development (especially databases are tricky).
I just happen to be making this one public.

STABILITY: This software is in development... so don''t use it
on huge projects with only one copy in existance. Aka, don''t
be stupid. However, for a new project, you can feel fairly secure
in its functionality.


==== INSTALLATION =====
To get the new plugin download from
http://rubyforge.org/frs/?group_id=1438&release_id=4838

Unzip so that... RAILS_PATH/vendor/plugins/devtower/

===== USE =====
A developer will run ''rake commit'' and DevTower will write the
database
 schema and data to file and clear out all the logs and then run ''svn
ci''.

Then, the next developer, to synchronize, will run ''rake
checkout'' and it
 will update his working directory and load up the new schema and new
 data fresh from the development server.


===== DOCUMENTATION =====
DevTower gives you the following commands:

rake db:hold:dump
     -writes the current database down to yaml files in
RAILS_APP/db/hold/*.yml Think of this as your data-dump to file.

rake db:hold:load
     -loads the yaml model files from db/hold/*.yml into the current
database. CHECK /db/hold BEFORE YOU DO THIS, because it will dump your
current db and load it. So, just make sure that the yaml files have
something in them.

rake commit:prepare
     -dumps the database down to the hold
     -dumps the schema to schema.rb
     -clears the /tmp folder
     -clears the /log files

rake commit:force
    -simply runs ''svn ci -m "#{diff of CHANGELOG}"''
    -uses whatever has been added to RAILS_APP/CHANGELOG since the last
commit as the commit message.

rake commit
    -runs commit:prepare
    -runs test ... your tests should pass before you commit anything ever!
    -runs commit:force to do the actual commit
    -uses whatever has been added to RAILS_APP/CHANGELOG since the last
commit as the commit message.

rake commit:skip_tests
    -if your a bad-boy and don''t have working tests, then prepare and
commit
anyway

rake checkout:after
    -WIPES current db
    -reads in schema from /db/schema.rb
    -reads in data from /db/hold/*.yml

rake checkout
    -performs ''svn up''
    -yes, i know its an update and not a checkout
    -runs checkout:after to do everything needed to start coding!


==== NOTES ====
Here are a few things that DevTower is *NOT*:

   * IT IS NOT a deployment system like Capistrano.
   * IT IS NOT the solution for everyone.
   * IT IS NOT built to work with CVS.
   * IT IS NOT completely stable.

All of those things being said. Here is what it does do.

  * Works with SVN.
  * Forces tests to pass before committing to the repository.
  * Writes database schema and data to /db/hold/*.yml before committing.
  * Reads them back after checkout.

===== BIG QUESTION ====
What about migrations?
    migrations are great when you are working with a stable/production
database
    and have muiltiple branches. but, we use schema.rb to keep things in
line early in
    development before branching and before stability... really good for
when the database is
     changingoften and radically. migrations are great for small changes,
painful for large
    ones very often.

Its not working right!
   That''s not a question. Stop yelling and go to
http://rubyforge.org/projects/devtower/
   and file a bug report.

It destroyed my precious, high-risk data! Why?
   I told you not to do it. Only use development data in the holds.

Does it work on windows?
   I don''t see why not. I didn''t test it. In fact, I only
tested it on
Ubuntu Dapper.

Does it work with Rails 1.0?
  No.

=====
Special thanks to Geoffrey Grosenbach for his ar_fixtures plugin.

-hampton catlin.

---------
http://www.justasloud.org
http://www.hampton2600.com
http://www.deadhobosociety.com  #for all your philosophical needs
http://rubyforge.org/projects/devtower/
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://wrath.rubyonrails.org/pipermail/rails/attachments/20060413/90714c97/attachment-0001.html

Just for the record: Thats absolutely not my experience. Migrations
work very well during all phases of the development if you

  a) Check in often
  b) Make lots of small migrations instead of one big.

tip: Lock a file in the repository when creating a migration until its
checked in. This will give your fellow developers a heads up that they
should wait for you.
> ===== BIG QUESTION ====>
> What about migrations?
>     migrations are great when you are working with a stable/production
> database
>     and have muiltiple branches. but, we use schema.rb to keep things in
> line early in
>     development before branching and before stability... really good for
> when the database is
>      changingoften and radically. migrations are great for small changes,
> painful for large
>     ones very often.

Is there any reason that some migrations couldn''t be generated
automatically?  Starting with a schema.rb of the current state of the
database on disk, you could then modify your database directly, and
run a script to dump out the schema and compare it to the one already
on disk, and generate a migration that describes the difference.  I
think this would only be useful for adding tables and columns
(renaming or modifying really wouldn''t work), but I''d imagine
that is
the most common type of migration anyway.

I like all the things that migrations give me, but I miss being able
to dig around in the db and make quick changes on the fly, in the
context of the rest of of the tables and columns that already exist. 
This had been a big selling point of rails.

- Isaac

On 4/13/06, Tobias L?tke <tobias.luetke@gmail.com>
wrote:
> Just for the record: Thats absolutely not my experience. Migrations
> work very well during all phases of the development if you
>
>   a) Check in often
>   b) Make lots of small migrations instead of one big.
>
> tip: Lock a file in the repository when creating a migration until its
> checked in. This will give your fellow developers a heads up that they
> should wait for you.
>
> > ===== BIG QUESTION ====> >
> > What about migrations?
> >     migrations are great when you are working with a stable/production
> > database
> >     and have muiltiple branches. but, we use schema.rb to keep things
in
> > line early in
> >     development before branching and before stability... really good
for
> > when the database is
> >      changingoften and radically. migrations are great for small
changes,
> > painful for large
> >     ones very often.
> _______________________________________________
> Rails mailing list
> Rails@lists.rubyonrails.org
> http://lists.rubyonrails.org/mailman/listinfo/rails
>
>

Tobias,

The main issue here is.... when I add a row called "created_on" to my
user
table and I make a migration for it. Then, my fellow developer grabs the new
source, and runs the migration. Then, he has a broken data-set. None of the
created_ons in his user are filled out!

There is only merging of structure, not data.

So, tell me how doing small migrations would keep that from happening?

Later on when we have a production server, we''d write stuff into our
migration to generate the correct data, because its worth taking the time to
do that to migrate an existing stable database.

The other issue here is when you have a large set of data. For instance, I
was working on a system for keeping events data. Well, I would enter 5
events as I was developing one day, then my partner-in-crime would enter 4
on hers... We both wanted that shared information to be merged.

So, this provides a serialized database in its entirety. Also, what if I do
early development on sqlite and want to move *all* of my existing data over
to some other database type. This system would use database serialization to
facilitate that.

Last comment, lots of people will find this horrible un-useful to their
work-habits. However, using migrations early on often left me and my other
developers with odd looking database datasets that would break and were a
pain in the neck to keep up with all the changes that were happening. So,
this certainly isn''t for everyone, but it is something that others who
develop in a style similar to mine might find useful.

-hampton.

On 4/13/06, Tobias L?tke <tobias.luetke@gmail.com>
wrote:
>
> Just for the record: Thats absolutely not my experience. Migrations
> work very well during all phases of the development if you
>
>   a) Check in often
>   b) Make lots of small migrations instead of one big.
>
> tip: Lock a file in the repository when creating a migration until its
> checked in. This will give your fellow developers a heads up that they
> should wait for you.
>
> > ===== BIG QUESTION ====> >
> > What about migrations?
> >     migrations are great when you are working with a stable/production
> > database
> >     and have muiltiple branches. but, we use schema.rb to keep things
in
> > line early in
> >     development before branching and before stability... really good
for
> > when the database is
> >      changingoften and radically. migrations are great for small
> changes,
> > painful for large
> >     ones very often.
> _______________________________________________
> Rails mailing list
> Rails@lists.rubyonrails.org
> http://lists.rubyonrails.org/mailman/listinfo/rails
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://wrath.rubyonrails.org/pipermail/rails/attachments/20060413/09215285/attachment.html

Migrations are ruby code which runs in your rails environment. Data is
meant to be converted and adjusted by them. In fact of Shopify''s 146
migrations only about 100 actually change the schema.rb. Most deal
with data transformation. Shopify is not in production mode yet ( well
it''s going to be later today )

On 4/13/06, Hampton <hcatlin@gmail.com> wrote:
> Tobias,
>
> The main issue here is.... when I add a row called "created_on"
to my user
> table and I make a migration for it. Then, my fellow developer grabs the
new
> source, and runs the migration. Then, he has a broken data-set. None of the
> created_ons in his user are filled out!
User.find(:all).each { |u| u.update_attribute(:created_on,
Time.parse("#{rand(27)+1}-#{rand(11)+1}-2005"))  }

cheers.

>
> There is only merging of structure, not data.
>
> So, tell me how doing small migrations would keep that from happening?
>
> Later on when we have a production server, we''d write stuff into
our
> migration to generate the correct data, because its worth taking the time
to
> do that to migrate an existing stable database.
>
> The other issue here is when you have a large set of data. For instance, I
> was working on a system for keeping events data. Well, I would enter 5
> events as I was developing one day, then my partner-in-crime would enter 4
> on hers... We both wanted that shared information to be merged.
>
> So, this provides a serialized database in its entirety. Also, what if I do
> early development on sqlite and want to move *all* of my existing data over
> to some other database type. This system would use database serialization
to
> facilitate that.
>
> Last comment, lots of people will find this horrible un-useful to their
> work-habits. However, using migrations early on often left me and my other
> developers with odd looking database datasets that would break and were a
> pain in the neck to keep up with all the changes that were happening. So,
> this certainly isn''t for everyone, but it is something that others
who
> develop in a style similar to mine might find useful.
>
> -hampton.
>
>
> On 4/13/06, Tobias L?tke <tobias.luetke@gmail.com> wrote:
> >
>  Just for the record: Thats absolutely not my experience. Migrations
> work very well during all phases of the development if you
>
>   a) Check in often
>   b) Make lots of small migrations instead of one big.
>
>  tip: Lock a file in the repository when creating a migration until its
> checked in. This will give your fellow developers a heads up that they
> should wait for you.
>
> > ===== BIG QUESTION ====> >
> > What about migrations?
> >     migrations are great when you are working with a stable/production
> > database
> >     and have muiltiple branches. but, we use schema.rb to keep things
in
> > line early in
> >     development before branching and before stability... really good
for
> > when the database is
> >      changingoften and radically. migrations are great for small
changes,
> > painful for large
> >     ones very often.
> _______________________________________________
>  Rails mailing list
> Rails@lists.rubyonrails.org
> http://lists.rubyonrails.org/mailman/listinfo/rails
>
>
>
> _______________________________________________
> Rails mailing list
> Rails@lists.rubyonrails.org
> http://lists.rubyonrails.org/mailman/listinfo/rails
>
>
>

--
Tobi
http://shopify.com       - modern e-commerce software
http://typo.leetsoft.com - Open source weblog engine
http://blog.leetsoft.com - Technical weblog

Maybe its just me but.... 146 migrations doesn''t exactly sound
efficient or
fun to do a checkout with.

The other advantage to using the "hold" with YAML files is that you
can
always take these YAML files and put them in your fixtures and run your
tests on your actual data instead of writing it by hand.


On 4/13/06, Tobias L?tke <tobias.luetke@gmail.com>
wrote:
>
> Migrations are ruby code which runs in your rails environment. Data is
> meant to be converted and adjusted by them. In fact of Shopify''s
146
> migrations only about 100 actually change the schema.rb. Most deal
> with data transformation. Shopify is not in production mode yet ( well
> it''s going to be later today )
>
> On 4/13/06, Hampton <hcatlin@gmail.com> wrote:
> > Tobias,
> >
> > The main issue here is.... when I add a row called
"created_on" to my
> user
> > table and I make a migration for it. Then, my fellow developer grabs
the
> new
> > source, and runs the migration. Then, he has a broken data-set. None
of
> the
> > created_ons in his user are filled out!
>
> User.find(:all).each { |u| u.update_attribute(:created_on,
> Time.parse("#{rand(27)+1}-#{rand(11)+1}-2005"))  }
>
> cheers.
>
>
> >
> > There is only merging of structure, not data.
> >
> > So, tell me how doing small migrations would keep that from happening?
> >
> > Later on when we have a production server, we''d write stuff
into our
> > migration to generate the correct data, because its worth taking the
> time to
> > do that to migrate an existing stable database.
> >
> > The other issue here is when you have a large set of data. For
instance,
> I
> > was working on a system for keeping events data. Well, I would enter 5
> > events as I was developing one day, then my partner-in-crime would
enter
> 4
> > on hers... We both wanted that shared information to be merged.
> >
> > So, this provides a serialized database in its entirety. Also, what if
I
> do
> > early development on sqlite and want to move *all* of my existing data
> over
> > to some other database type. This system would use database
> serialization to
> > facilitate that.
> >
> > Last comment, lots of people will find this horrible un-useful to
their
> > work-habits. However, using migrations early on often left me and my
> other
> > developers with odd looking database datasets that would break and
were
> a
> > pain in the neck to keep up with all the changes that were happening.
> So,
> > this certainly isn''t for everyone, but it is something that
others who
> > develop in a style similar to mine might find useful.
> >
> > -hampton.
> >
> >
> > On 4/13/06, Tobias L?tke <tobias.luetke@gmail.com> wrote:
> > >
> >  Just for the record: Thats absolutely not my experience. Migrations
> > work very well during all phases of the development if you
> >
> >   a) Check in often
> >   b) Make lots of small migrations instead of one big.
> >
> >  tip: Lock a file in the repository when creating a migration until
its
> > checked in. This will give your fellow developers a heads up that they
> > should wait for you.
> >
> > > ===== BIG QUESTION ====> > >
> > > What about migrations?
> > >     migrations are great when you are working with a
stable/production
> > > database
> > >     and have muiltiple branches. but, we use schema.rb to keep
things
> in
> > > line early in
> > >     development before branching and before stability... really
good
> for
> > > when the database is
> > >      changingoften and radically. migrations are great for small
> changes,
> > > painful for large
> > >     ones very often.
> > _______________________________________________
> >  Rails mailing list
> > Rails@lists.rubyonrails.org
> > http://lists.rubyonrails.org/mailman/listinfo/rails
> >
> >
> >
> > _______________________________________________
> > Rails mailing list
> > Rails@lists.rubyonrails.org
> > http://lists.rubyonrails.org/mailman/listinfo/rails
> >
> >
> >
>
>
> --
> Tobi
> http://shopify.com       - modern e-commerce software
> http://typo.leetsoft.com - Open source weblog engine
> http://blog.leetsoft.com - Technical weblog
> _______________________________________________
> Rails mailing list
> Rails@lists.rubyonrails.org
> http://lists.rubyonrails.org/mailman/listinfo/rails
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://wrath.rubyonrails.org/pipermail/rails/attachments/20060413/ccf3b97f/attachment.html

Hampton-

	Pretty cool stuff. I will play with this a bit ans see if it fits in  
with our dev methodologies.

Thanks
-Ezra

On Apr 13, 2006, at 9:37 AM, Hampton wrote:
> DevTower Beta3! (0.3)
>
> I''m proud to announce some major changes and improvements to  
> DevTower in this beta release! First and foremost, DevTower is now  
> a full plugin. Also, DevTower works (only) with Rails 1.1.
>
>
> ==== ABOUT ====>
> DevTower used to synchronize the development of Ruby On
> Rails applications with developers working simultaniously
> on multiple systems with Subversion. DevTower is built
> around the philosophy of responsible group-development
> and enforces certain development practices which you may
> or may not like. However, it is my belief that all are good
> practices for any collaborative development system.
>
> It is comprised of a few extensions to ActiveRecord::Base
> (via a modified ar_fixtures) and several Rake tasks to speed
> group development.
>
> This was one of those things that was made for use in-house.
> Many of you probably have scripts around that do similar things
> to synchronize your development (especially databases are tricky).
> I just happen to be making this one public.
>
> STABILITY: This software is in development... so don''t use it
> on huge projects with only one copy in existance. Aka, don''t
> be stupid. However, for a new project, you can feel fairly secure
> in its functionality.
>
>
> ==== INSTALLATION =====>
> To get the new plugin download from http://rubyforge.org/frs/? 
> group_id=1438&release_id=4838
>
> Unzip so that... RAILS_PATH/vendor/plugins/devtower/
>
> ===== USE =====>
> A developer will run ''rake commit'' and DevTower will
write the
> database
>  schema and data to file and clear out all the logs and then run  
> ''svn ci''.
>
> Then, the next developer, to synchronize, will run ''rake
checkout''
> and it
>  will update his working directory and load up the new schema and new
>  data fresh from the development server.
>
>
> ===== DOCUMENTATION =====>
> DevTower gives you the following commands:
>
> rake db:hold:dump
>      -writes the current database down to yaml files in RAILS_APP/ 
> db/hold/*.yml Think of this as your data-dump to file.
>
> rake db:hold:load
>      -loads the yaml model files from db/hold/*.yml into the  
> current database. CHECK /db/hold BEFORE YOU DO THIS, because it  
> will dump your current db and load it. So, just make sure that the  
> yaml files have something in them.
>
> rake commit:prepare
>      -dumps the database down to the hold
>      -dumps the schema to schema.rb
>      -clears the /tmp folder
>      -clears the /log files
>
> rake commit:force
>     -simply runs ''svn ci -m "#{diff of
CHANGELOG}"''
>     -uses whatever has been added to RAILS_APP/CHANGELOG since the  
> last commit as the commit message.
>
> rake commit
>     -runs commit:prepare
>     -runs test ... your tests should pass before you commit  
> anything ever!
>     -runs commit:force to do the actual commit
>     -uses whatever has been added to RAILS_APP/CHANGELOG since the  
> last commit as the commit message.
>
> rake commit:skip_tests
>     -if your a bad-boy and don''t have working tests, then prepare
> and commit anyway
>
> rake checkout:after
>     -WIPES current db
>     -reads in schema from /db/schema.rb
>     -reads in data from /db/hold/*.yml
>
> rake checkout
>     -performs ''svn up''
>     -yes, i know its an update and not a checkout
>     -runs checkout:after to do everything needed to start coding!
>
>
> ==== NOTES ====>
> Here are a few things that DevTower is *NOT*:
>
>    * IT IS NOT a deployment system like Capistrano.
>    * IT IS NOT the solution for everyone.
>    * IT IS NOT built to work with CVS.
>    * IT IS NOT completely stable.
>
> All of those things being said. Here is what it does do.
>
>   * Works with SVN.
>   * Forces tests to pass before committing to the repository.
>   * Writes database schema and data to /db/hold/*.yml before  
> committing.
>   * Reads them back after checkout.
>
> ===== BIG QUESTION ====>
> What about migrations?
>     migrations are great when you are working with a stable/ 
> production database
>     and have muiltiple branches. but, we use schema.rb to keep  
> things in line early in
>     development before branching and before stability... really  
> good for when the database is
>      changingoften and radically. migrations are great for small  
> changes, painful for large
>     ones very often.
>
> Its not working right!
>    That''s not a question. Stop yelling and go to http:// 
> rubyforge.org/projects/devtower/
>    and file a bug report.
>
> It destroyed my precious, high-risk data! Why?
>    I told you not to do it. Only use development data in the holds.
>
> Does it work on windows?
>    I don''t see why not. I didn''t test it. In fact, I only
tested it
> on Ubuntu Dapper.
>
> Does it work with Rails 1.0?
>   No.
>
> =====>
> Special thanks to Geoffrey Grosenbach for his ar_fixtures plugin.
>
> -hampton catlin.
>
> ---------
> http://www.justasloud.org
> http://www.hampton2600.com
> http://www.deadhobosociety.com  #for all your philosophical needs
> http://rubyforge.org/projects/devtower/
> _______________________________________________
> Rails mailing list
> Rails@lists.rubyonrails.org
> http://lists.rubyonrails.org/mailman/listinfo/rails
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://wrath.rubyonrails.org/pipermail/rails/attachments/20060413/06947303/attachment-0001.html

Hampton wrote:
> Maybe its just me but.... 146 migrations doesn''t exactly sound
efficient
> or
> fun to do a checkout with.

Actually, sounds a lot better to me.  If they weren''t so stinking easy 
to do, the sort of stuff that gets done in a migration ends up piling up 
into a bigger database change.  That big database change then happens at 
some coordinated time when everyone can freeze what they''re doing and 
get an update.

Instead of 146 small changes, you have (say) 12 bigger changes.  Lots 
more can go wrong with big changes than small changes.

Ease of migrations allows them to become more atomic and cause smaller 
ripples throughout the design.

   Jake

-- 
Posted via http://www.ruby-forum.com/.

On 4/13/06, Hampton <hcatlin@gmail.com> wrote:
> I''m proud to announce some major changes and improvements to
DevTower in
> this beta release!
Hampton,

This is really cool!  =)  Working great for me so far.

Noticed a couple of small bugs (missing slash in hold path, quotes not
escaped in commit comments, and /n being inserted instead of \n (new
line)), and added code to not copy contents of the session table if
you are storing sessions in the database.  Diff attached.

Thanks!

- Isaac

--- vendor/plugins/devtower/lib/thehold.rb      Fri Apr 21 16:44:04 2006
+++ /src/devtower/lib/thehold.rb      Sat Apr  8 11:33:18 2006
@@ -11,7 +11,7 @@
     require ''active_record/fixtures''
       ActiveRecord::Base.establish_connection(RAILS_ENV.to_sym)
       Dir.glob(File.join(RAILS_ROOT, ''db'',
''hold'',
''*.{yml,csv}'')).each do |fixture_file|
-        Fixtures.create_fixtures(''db/hold'',
File.basename(fixture_file, ''.*''))
+        Fixtures.create_fixtures(''db/hold/'',
File.basename(fixture_file, ''.*''))
     end
   end

@@ -34,7 +34,7 @@
       connection.select_all("SELECT * FROM
#{join.options[:join_table]}").each_with_index { |record, i|
         hsh["join_#{i}"] = record
       }
-     
write_file(File.expand_path("db/hold#{join.options[:join_table]}.yml",
RAILS_ROOT), hsh.to_yaml)
+     
write_file(File.expand_path("db/hold/#{join.options[:join_table]}.yml",
RAILS_ROOT), hsh.to_yaml)
     end
   end

--- vendor/plugins/devtower/tasks/devtower.rake Sat Apr 22 03:40:21 2006
+++ /src/devtower/tasks/devtower.rake Sat Apr  8 10:41:49 2006
@@ -8,6 +8,7 @@
   def models
     Dir.glob(RAILS_ROOT + ''/app/models/*.rb'').each { |file|
require file }
     models = Object.subclasses_of(ActiveRecord::Base)
+    models.delete(CGI::Session::ActiveRecordStore::Session)  # no
sessions for us!
     puts models.inspect
     models
   end
@@ -79,12 +80,12 @@
     #generate a commit message from the lines added in the file CHANGELOG
     result = `svn diff CHANGELOG`.scan(/^[+].*$/).map { |str|
        unless str[1]==43
-               str[1,str.length-1] + "/n"
+               str[1,str.length-1] + "\n"
        end
     }
     message = result.join()

-    `svn ci -m "#{message}"`
+    `svn ci -m
"#{message.gsub(''"'',''\"'')}"`
   end

   desc "Dumps the schema, its data to hold, and clears the logs."