THE SCHEMA IN THE MODEL a small write up on ''putting the schema in the model'' This is a write up on an issue best covered in a mailing list thread of Januari 2006 (see the links in the text), I repost it because I think it deserves a place on the agenda. == Why? =I was switching back and forward between the model files and the schema.rb -- off course I have "active_record.schema_format = :ruby" in my enviroment -- and naturally started thinking the following: "Why is the model specified in two separate places?" "Shouldn''t the model files be the only place to do model stuff?" While this question was mainly because I was fed up switching, I kind of felt that I was at something: that we should spread stuff over more files/locations than stricktly nessecary. Maybe it can be called "Don''t Spread Yourself" (DSY). == Some research =Quickly I asked Mr Google several questions on this topic, and he came up with quite a few insights on the "schema in the model" utopia I was dreaming of. First of all I learned the historic background of ''not having the schema in the model''. Originally the reason for not putting the schema in the model was because of "Don''t Repeat Yourself" (DRY); the model had to be specified only once, in the early rails days that was in the database itself. Can you imagine? Directly modifying the database? Yes dear readers, since the salvation by "migrations" we almost forgot about it, but in the early days of rails development there where brave coders directly modifying databases. I''m kidding, but the point i want to make is that the arrival of "migrations" and the "active_record.schema_format = :ruby" option already allows us -- but does not force us -- to specify the database schema in nice ruby code in stead of in the databes it self. The fact that these new, higher levels of control are not hiding any stuff below is what i consider one of the powers of rails: You _can_ use all the nice sugar that makes it a bliss to develop on: but it is not you to use them -- you''re always welcome to get your hands dirty, digg deep, and direclty write SQL, HTML, and (recently) JavaScript if you want/have/need to. Maybe we can call this: "Don''t Hide Lower Levels" (DHLL). So since migrations and the "active_record.schema_format = :ruby" option we don''t have to specify the model in the database, but more important: setting up and modifying the database schema can now be database agnostic, distributable and versionable. Migrations allow us a higer level of abstaction. Yet we can (IMHO) still go higher on the ladder of abstraction, closer to the mythical "Don''t Repeat Yourself" goal, and closer to the "Don''t Spread Yourself" goal i just came up with. This because at the current level of abstraction (using migrations) we are: 1. repeating ourself when filling the up() and down() methods of a migration 2. spreading the model over more than one location, namely: the model file, the migrations (write only) and the schema.rb file (read only) Didn''t you at one point, when using migrations, think of a way to automate the filling of the up() and down() methods -- doesn''t filling these methods feel like repeating yourself? Yet most likely it did not hurt enough to actually change it, since you''re usually only writing a little change in a migration. == What could it look like =My ideas materialized when it read this thread: http://lists.rubyonrails.org/pipermail/rails/2006-January/009640.html On that thread Jules posts, amoung other things, a beautiful example of what "putting the schema in a model" could look like: --------------------- class Author < ActiveRecord::Base has_many :posts attribute :name, :string attribute :email, :string do |a| a.validate_format :with => /regex/ end end class Post < ActiveRecord::Base belongs_to :author # automatically adds author_id to the table attribute :title, :string do |a| a.validate_presence a.validate_length :in => 3..100 end attribute :summary, :text attribute :content, :text end --------------------- [from: http://lists.rubyonrails.org/pipermail/rails/2006-January/011325.html] Then Jon Smirl comes up with a list of what needs to be done to make migrations play along schema''s in the model: http://lists.rubyonrails.org/pipermail/rails/2006-January/011395.html If I understand it correctly Jon suggests that the schema changes one makes in the model file should be used to generate migrations, that are in their turn used to (automatically or not) propagate the changes to the database. Jon also makes (in that first and in later posts) some more interesting notes on how this should be implemented. Not much later the thread stops. == Now what? =I totally lack the skills and experience to write a patch for this feature, I dont even know if this is a feature that the rails community can agree upon (when searching the web I found also quite some opposition towards ''putting the schema in the model''). Usually a relatively big patch like this would ideally be implemented as a plugin first. While I know Ruby is highly dynamic, I don''t know if it is possible to implement such a feature as plugin. By explaining and reposting this issue i hope that: 1. a discussion on the validity of ''the schema in the model'' approach 2. some more issues regarding the implementation of this feature can be straightend out. Thank you for reading, thanks to the Rails community, these where my .2EUR Cies Breijs. -- "Computer games don''t affect kids; I mean if Pac-Man affected us as kids, we''d all be running around in darkened rooms, munching magic pills and listening to repetitive electronic music." -- Kristian Wilson (Nintendo, Inc), 1989
THE SCHEMA IN THE MODEL a small write up on ''putting the schema in the model'' This is a write up on an issue best covered in a mailing list thread of Januari 2006 (see the links in the text), I repost it because I think it deserves a place on the agenda. == Why? =I was switching back and forward between the model files and the schema.rb -- off course I have "active_record.schema_format = :ruby" in my enviroment -- and naturally started thinking the following: "Why is the model specified in two separate places?" "Shouldn''t the model files be the only place to do model stuff?" While this question was mainly because I was fed up switching, I kind of felt that I was at something: that we should spread stuff over more files/locations than stricktly nessecary. Maybe it can be called "Don''t Spread Yourself" (DSY). == Some research =Quickly I asked Mr Google several questions on this topic, and he came up with quite a few insights on the "schema in the model" utopia I was dreaming of. First of all I learned the historic background of ''not having the schema in the model''. Originally the reason for not putting the schema in the model was because of "Don''t Repeat Yourself" (DRY); the model had to be specified only once, in the early rails days that was in the database itself. Can you imagine? Directly modifying the database? Yes dear readers, since the salvation by "migrations" we almost forgot about it, but in the early days of rails development there where brave coders directly modifying databases. I''m kidding, but the point i want to make is that the arrival of "migrations" and the "active_record.schema_format = :ruby" option already allows us -- but does not force us -- to specify the database schema in nice ruby code in stead of in the databes it self. The fact that these new, higher levels of control are not hiding any stuff below is what i consider one of the powers of rails: You _can_ use all the nice sugar that makes it a bliss to develop on: but it is not you to use them -- you''re always welcome to get your hands dirty, digg deep, and direclty write SQL, HTML, and (recently) JavaScript if you want/have/need to. Maybe we can call this: "Don''t Hide Lower Levels" (DHLL). So since migrations and the "active_record.schema_format = :ruby" option we don''t have to specify the model in the database, but more important: setting up and modifying the database schema can now be database agnostic, distributable and versionable. Migrations allow us a higer level of abstaction. Yet we can (IMHO) still go higher on the ladder of abstraction, closer to the mythical "Don''t Repeat Yourself" goal, and closer to the "Don''t Spread Yourself" goal i just came up with. This because at the current level of abstraction (using migrations) we are: 1. repeating ourself when filling the up() and down() methods of a migration 2. spreading the model over more than one location, namely: the model file, the migrations (write only) and the schema.rb file (read only) Didn''t you at one point, when using migrations, think of a way to automate the filling of the up() and down() methods -- doesn''t filling these methods feel like repeating yourself? Yet most likely it did not hurt enough to actually change it, since you''re usually only writing a little change in a migration. == What could it look like =My ideas materialized when it read this thread: http://lists.rubyonrails.org/pipermail/rails/2006-January/009640.html On that thread Jules posts, amoung other things, a beautiful example of what "putting the schema in a model" could look like: --------------------- class Author < ActiveRecord::Base has_many :posts attribute :name, :string attribute :email, :string do |a| a.validate_format :with => /regex/ end end class Post < ActiveRecord::Base belongs_to :author # automatically adds author_id to the table attribute :title, :string do |a| a.validate_presence a.validate_length :in => 3..100 end attribute :summary, :text attribute :content, :text end --------------------- [from: http://lists.rubyonrails.org/pipermail/rails/2006-January/011325.html] Then Jon Smirl comes up with a list of what needs to be done to make migrations play along schema''s in the model: http://lists.rubyonrails.org/pipermail/rails/2006-January/011395.html If I understand it correctly Jon suggests that the schema changes one makes in the model file should be used to generate migrations, that are in their turn used to (automatically or not) propagate the changes to the database. Jon also makes (in that first and in later posts) some more interesting notes on how this should be implemented. Not much later the thread stops. == Now what? =I totally lack the skills and experience to write a patch for this feature, I dont even know if this is a feature that the rails community can agree upon (when searching the web I found also quite some opposition towards ''putting the schema in the model''). Usually a relatively big patch like this would ideally be implemented as a plugin first. While I know Ruby is highly dynamic, I don''t know if it is possible to implement such a feature as plugin. By explaining and reposting this issue i hope that: 1. a discussion on the validity of ''the schema in the model'' approach 2. some more issues regarding the implementation of this feature can be straightend out. Thank you for reading, thanks to the Rails community, these where my .2EUR Cies Breijs. -- "Computer games don''t affect kids; I mean if Pac-Man affected us as kids, we''d all be running around in darkened rooms, munching magic pills and listening to repetitive electronic music." -- Kristian Wilson (Nintendo, Inc), 1989 -- "Computer games don''t affect kids; I mean if Pac-Man affected us as kids, we''d all be running around in darkened rooms, munching magic pills and listening to repetitive electronic music." -- Kristian Wilson (Nintendo, Inc), 1989
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi Cies, I''m glad you renewed this discussion. IMHO this is one of the best examples of un-DRY behavior in rails that I''ve seen. More comments below.> == What could it look like => My ideas materialized when it read this thread: > http://lists.rubyonrails.org/pipermail/rails/2006-January/009640.html > > On that thread Jules posts, amoung other things, a beautiful example > of what "putting the schema in a model" could look like: > > --------------------- > class Author < ActiveRecord::Base > has_many :posts > > attribute :name, :string > > attribute :email, :string do |a| > a.validate_format :with => /regex/ > end > end > > > class Post < ActiveRecord::Base > belongs_to :author # automatically adds author_id to the table > > attribute :title, :string do |a| > a.validate_presence > a.validate_length :in => 3..100 > end > > attribute :summary, :text > attribute :content, :text > end > --------------------- > [from: http://lists.rubyonrails.org/pipermail/rails/2006-January/ > 011325.html]I wrote a plugin that allows this syntax now: http://lists.rubyonrails.org/pipermail/rails/2006-April/037180.html It doesn''t deal with the migration side of the issue, just DRYing up the validation rule specification. My plan was to have all the validation rules in one place, with enough data to generate the migrations from the model files later on.> I totally lack the skills and experience to write a patch for this > feature, I dont even know if this is a feature that the rails > community can agree upon (when searching the web I found also quite > some opposition towards ''putting the schema in the model'').It would be interesting for you to post links to the arguments against the idea so that we may discuss them. I think we can all agree that we have to specify the schema someplace at least once. Traditionally rails has used the database as the single reference point, and the models are generated from the current state of the database tables. There has been talk of generating validation rules from the table column definitions, and using foreign keys to auto-generate the model relationships. The problem with this approach is that every database has different capabilities, some have a really rich column-level constraints and foreign keys, while others are relatively weak in this area like SQLite and older versions of MySQL. If you used this approach and developed on one database you''d be forever tied to that database. Rails is meant to be database agnostic, so it cannot rely on the database as being the single reference point for much more than just the column names, which leaves migrations or the model as being the only reliable place to define relationships and validation rules. In Rails you have to specify everything twice: once in the migration and once in the model. Effort has to be taken to keep them in sync, so that the validates_length_of matches the :string column''s length for example. You have to define configuration for the database, and the configuration for the models -- with both needing to be in agreement otherwise you''re risking introducing bugs. This continual double configuration leads to a very un-DRY experience. Given the choice of these two places to define relationships and validation, I''d much rather have it in the model and generate the migration from the current model. The model allows a much richer set of constraints to be used than a database could ever hope to have. I''m all for communicating a subset of those constraints to the database when performing a migration -- send as much as the database can understand, but the model should remain the single point of reference. - -- Thanks, Dan __________________________________________________________________ Dan Kubb Autopilot Marketing Inc. Email: dan.kubb@autopilotmarketing.com Phone: 1 (604) 820-0212 Web: http://autopilotmarketing.com/ vCard: http://autopilotmarketing.com/~dan.kubb/vcard __________________________________________________________________ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (Darwin) iD8DBQFEcx6m4DfZD7OEWk0RAlOUAJ96uOEJaN5boOFWrA+SA5DPTXxA8wCgme2e y7wYo7A8oNkhOBK6qNB9/wM=w0N9 -----END PGP SIGNATURE-----
Dan,> I wrote a plugin that allows this syntax now: > > http://lists.rubyonrails.org/pipermail/rails/2006-April/037180.html > > It doesn''t deal with the migration side of the issue, just DRYing up the > validation rule specification. My plan was to have all the validation > rules in one place, with enough data to generate the migrations > from the model files later on.Nice! The syntax looks like a DRY-bliss to work with, i''m trying it right now. Quote of some of the plugins syntax: ---------------------- class Country < ActiveRecord::Base attribute :name, :string do |a| a.validates_length :within => 1..48 a.validates_format :with => /regexp/ a.validates_uniqueness end attribute :code_alpha2, :string do |a| a.validates_length :is => 2 a.validates_format :with => /\A[A-Z]+\z/ a.validates_uniqueness end # [...] end ----------------------> > I totally lack the skills and experience to write a patch for this > > feature, I dont even know if this is a feature that the rails > > community can agree upon (when searching the web I found also quite > > some opposition towards ''putting the schema in the model''). > > It would be interesting for you to post links to the arguments against > the idea so that we may discuss them.I tried but I could not find it anymore. I remember that the main argument against was that "the schema belongs IN the database". In my inital post to this thread I also try to address this point by explaining that with migrations the schema is no longer ''only'' in the database.> I think we can all agree that we have to specify the schema someplace > at least once.Yeah, and most desireably: only once. No repeating of identifiers, symbols and mappings... please ;)> Traditionally rails has used the database as the single > reference point, and the models are generated from the current state of > the database tables. There has been talk of generating validation > rules from the table column definitions, and using foreign keys to > auto-generate the model relationships. The problem with this approach > is that every database has different capabilities, some have a really > rich column-level constraints and foreign keys, while others are > relatively > weak in this area like SQLite and older versions of MySQL. If you used > this approach and developed on one database you''d be forever tied > to that database. Rails is meant to be database agnostic, so it cannot > rely on the database as being the single reference point for much more > than just the column names, which leaves migrations or the model as > being > the only reliable place to define relationships and validation rules.Well the nice thing about Rails is that it can be db agnostic (though migrations), yet you also could use all features of your specific database and not use the migration. The developer has to make a trade-off between (in this case): - developing speed, portability, managability, and - database optimalisation Personally, for my projects I use the database as quick/dumb/structured storage. So no need for db optimalisations -- I happily use migrations to manage my db schema''s.> In Rails you have to specify everything twice: once in the migration > and once in the model. Effort has to be taken to keep them in sync, > so that the validates_length_of matches the :string column''s length > for example. You have to define configuration for the database, > and the configuration for the models -- with both needing to be in > agreement otherwise you''re risking introducing bugs. This continual > double configuration leads to a very un-DRY experience.Yep, I totally agree... Allthough I also understand the historics behind it: the dark days before migrations.> Given the choice of these two places to define relationships and > validation, I''d much rather have it in the model and generate the > migration from the current model. The model allows a much richer set > of constraints to be used than a database could ever hope to have. > I''m all for communicating a subset of those constraints to the > database when performing a migration -- send as much as the database > can understand, but the model should remain the single point > of reference.Here I differ slightly from your point of view. The developer should be able to choose whether he wants directly use the database, or that he likes to use highlevel stuff like migratons, your ActiveAttribute plugin or a future technique that hooks ActiveAttribute up with migrations. Rails should be flexible in this. Again: I, and probably many others using the db as dumb storage, would love to put all model stuff in the model files.> Thanks,Thank you Dan for the ActiveAttribute plugin! Cies Breijs. -- "Computer games don''t affect kids; I mean if Pac-Man affected us as kids, we''d all be running around in darkened rooms, munching magic pills and listening to repetitive electronic music." -- Kristian Wilson (Nintendo, Inc), 1989
I''ve thought of that before. Ok, so it''s never occured to me that up() and down() is against DRY (but you are right), I have wondered why I can''t just define the model along with the migrations myself. I guess a plugin like that could be my project over the summer (I''m a student, you see)... it sure would get me some good experience in Ruby and rails. It would just mean having some uber-Daddy class which uses methods from both ActiveRecord and the schema one (the name of which I can''t remember and I''m not at my rails computer)... I really don''t evision it being too hard at all. Of course, now that I''ve said this, someone might beat me to it. I can''t see something like that making it into the rails framework because, as you say, there''s a lot of opposition, but that doesn''t bother me... a plugin is just as good. -N On 23/05/06, cies <cies.breijs@gmail.com> wrote:> Dan, > > > I wrote a plugin that allows this syntax now: > > > > http://lists.rubyonrails.org/pipermail/rails/2006-April/037180.html > > > > It doesn''t deal with the migration side of the issue, just DRYing up the > > validation rule specification. My plan was to have all the validation > > rules in one place, with enough data to generate the migrations > > from the model files later on. > > Nice! > > The syntax looks like a DRY-bliss to work with, i''m trying it right now. > > Quote of some of the plugins syntax: > ---------------------- > class Country < ActiveRecord::Base > attribute :name, :string do |a| > a.validates_length :within => 1..48 > a.validates_format :with => /regexp/ > a.validates_uniqueness > end > > attribute :code_alpha2, :string do |a| > a.validates_length :is => 2 > a.validates_format :with => /\A[A-Z]+\z/ > a.validates_uniqueness > end > > # [...] > end > ---------------------- > > > > > I totally lack the skills and experience to write a patch for this > > > feature, I dont even know if this is a feature that the rails > > > community can agree upon (when searching the web I found also quite > > > some opposition towards ''putting the schema in the model''). > > > > It would be interesting for you to post links to the arguments against > > the idea so that we may discuss them. > > I tried but I could not find it anymore. I remember that the main > argument against was that "the schema belongs IN the database". In my > inital post to this thread I also try to address this point by > explaining that with migrations the schema is no longer ''only'' in the > database. > > > > I think we can all agree that we have to specify the schema someplace > > at least once. > > Yeah, and most desireably: only once. No repeating of identifiers, > symbols and mappings... please ;) > > > Traditionally rails has used the database as the single > > reference point, and the models are generated from the current state of > > the database tables. There has been talk of generating validation > > rules from the table column definitions, and using foreign keys to > > auto-generate the model relationships. The problem with this approach > > is that every database has different capabilities, some have a really > > rich column-level constraints and foreign keys, while others are > > relatively > > weak in this area like SQLite and older versions of MySQL. If you used > > this approach and developed on one database you''d be forever tied > > to that database. Rails is meant to be database agnostic, so it cannot > > rely on the database as being the single reference point for much more > > than just the column names, which leaves migrations or the model as > > being > > the only reliable place to define relationships and validation rules. > > Well the nice thing about Rails is that it can be db agnostic (though > migrations), yet you also could use all features of your specific > database and not use the migration. The developer has to make a > trade-off between (in this case): > - developing speed, portability, managability, and > - database optimalisation > > Personally, for my projects I use the database as > quick/dumb/structured storage. So no need for db optimalisations -- I > happily use migrations to manage my db schema''s. > > > > In Rails you have to specify everything twice: once in the migration > > and once in the model. Effort has to be taken to keep them in sync, > > so that the validates_length_of matches the :string column''s length > > for example. You have to define configuration for the database, > > and the configuration for the models -- with both needing to be in > > agreement otherwise you''re risking introducing bugs. This continual > > double configuration leads to a very un-DRY experience. > > Yep, I totally agree... Allthough I also understand the historics > behind it: the dark days before migrations. > > > > Given the choice of these two places to define relationships and > > validation, I''d much rather have it in the model and generate the > > migration from the current model. The model allows a much richer set > > of constraints to be used than a database could ever hope to have. > > I''m all for communicating a subset of those constraints to the > > database when performing a migration -- send as much as the database > > can understand, but the model should remain the single point > > of reference. > > Here I differ slightly from your point of view. The developer should > be able to choose whether he wants directly use the database, or that > he likes to use highlevel stuff like migratons, your ActiveAttribute > plugin or a future technique that hooks ActiveAttribute up with > migrations. Rails should be flexible in this. > Again: I, and probably many others using the db as dumb storage, would > love to put all model stuff in the model files. > > > > Thanks, > Thank you Dan for the ActiveAttribute plugin! > > Cies Breijs. > > > > -- > "Computer games don''t affect kids; I mean if Pac-Man affected us as > kids, we''d all be running around in darkened rooms, munching magic > pills and listening to repetitive electronic music." -- Kristian > Wilson (Nintendo, Inc), 1989 > _______________________________________________ > Rails mailing list > Rails@lists.rubyonrails.org > http://lists.rubyonrails.org/mailman/listinfo/rails >
Oh, ok. I even got beaten to the post... *red face* On 23/05/06, njmacinnes@gmail.com <njmacinnes@gmail.com> wrote:> I''ve thought of that before. Ok, so it''s never occured to me that up() > and down() is against DRY (but you are right), I have wondered why I > can''t just define the model along with the migrations myself. I guess > a plugin like that could be my project over the summer (I''m a student, > you see)... it sure would get me some good experience in Ruby and > rails. It would just mean having some uber-Daddy class which uses > methods from both ActiveRecord and the schema one (the name of which I > can''t remember and I''m not at my rails computer)... I really don''t > evision it being too hard at all. Of course, now that I''ve said this, > someone might beat me to it. > > I can''t see something like that making it into the rails framework > because, as you say, there''s a lot of opposition, but that doesn''t > bother me... a plugin is just as good. > > -N > > On 23/05/06, cies <cies.breijs@gmail.com> wrote: > > Dan, > > > > > I wrote a plugin that allows this syntax now: > > > > > > http://lists.rubyonrails.org/pipermail/rails/2006-April/037180.html > > > > > > It doesn''t deal with the migration side of the issue, just DRYing up the > > > validation rule specification. My plan was to have all the validation > > > rules in one place, with enough data to generate the migrations > > > from the model files later on. > > > > Nice! > > > > The syntax looks like a DRY-bliss to work with, i''m trying it right now. > > > > Quote of some of the plugins syntax: > > ---------------------- > > class Country < ActiveRecord::Base > > attribute :name, :string do |a| > > a.validates_length :within => 1..48 > > a.validates_format :with => /regexp/ > > a.validates_uniqueness > > end > > > > attribute :code_alpha2, :string do |a| > > a.validates_length :is => 2 > > a.validates_format :with => /\A[A-Z]+\z/ > > a.validates_uniqueness > > end > > > > # [...] > > end > > ---------------------- > > > > > > > > I totally lack the skills and experience to write a patch for this > > > > feature, I dont even know if this is a feature that the rails > > > > community can agree upon (when searching the web I found also quite > > > > some opposition towards ''putting the schema in the model''). > > > > > > It would be interesting for you to post links to the arguments against > > > the idea so that we may discuss them. > > > > I tried but I could not find it anymore. I remember that the main > > argument against was that "the schema belongs IN the database". In my > > inital post to this thread I also try to address this point by > > explaining that with migrations the schema is no longer ''only'' in the > > database. > > > > > > > I think we can all agree that we have to specify the schema someplace > > > at least once. > > > > Yeah, and most desireably: only once. No repeating of identifiers, > > symbols and mappings... please ;) > > > > > Traditionally rails has used the database as the single > > > reference point, and the models are generated from the current state of > > > the database tables. There has been talk of generating validation > > > rules from the table column definitions, and using foreign keys to > > > auto-generate the model relationships. The problem with this approach > > > is that every database has different capabilities, some have a really > > > rich column-level constraints and foreign keys, while others are > > > relatively > > > weak in this area like SQLite and older versions of MySQL. If you used > > > this approach and developed on one database you''d be forever tied > > > to that database. Rails is meant to be database agnostic, so it cannot > > > rely on the database as being the single reference point for much more > > > than just the column names, which leaves migrations or the model as > > > being > > > the only reliable place to define relationships and validation rules. > > > > Well the nice thing about Rails is that it can be db agnostic (though > > migrations), yet you also could use all features of your specific > > database and not use the migration. The developer has to make a > > trade-off between (in this case): > > - developing speed, portability, managability, and > > - database optimalisation > > > > Personally, for my projects I use the database as > > quick/dumb/structured storage. So no need for db optimalisations -- I > > happily use migrations to manage my db schema''s. > > > > > > > In Rails you have to specify everything twice: once in the migration > > > and once in the model. Effort has to be taken to keep them in sync, > > > so that the validates_length_of matches the :string column''s length > > > for example. You have to define configuration for the database, > > > and the configuration for the models -- with both needing to be in > > > agreement otherwise you''re risking introducing bugs. This continual > > > double configuration leads to a very un-DRY experience. > > > > Yep, I totally agree... Allthough I also understand the historics > > behind it: the dark days before migrations. > > > > > > > Given the choice of these two places to define relationships and > > > validation, I''d much rather have it in the model and generate the > > > migration from the current model. The model allows a much richer set > > > of constraints to be used than a database could ever hope to have. > > > I''m all for communicating a subset of those constraints to the > > > database when performing a migration -- send as much as the database > > > can understand, but the model should remain the single point > > > of reference. > > > > Here I differ slightly from your point of view. The developer should > > be able to choose whether he wants directly use the database, or that > > he likes to use highlevel stuff like migratons, your ActiveAttribute > > plugin or a future technique that hooks ActiveAttribute up with > > migrations. Rails should be flexible in this. > > Again: I, and probably many others using the db as dumb storage, would > > love to put all model stuff in the model files. > > > > > > > Thanks, > > Thank you Dan for the ActiveAttribute plugin! > > > > Cies Breijs. > > > > > > > > -- > > "Computer games don''t affect kids; I mean if Pac-Man affected us as > > kids, we''d all be running around in darkened rooms, munching magic > > pills and listening to repetitive electronic music." -- Kristian > > Wilson (Nintendo, Inc), 1989 > > _______________________________________________ > > Rails mailing list > > Rails@lists.rubyonrails.org > > http://lists.rubyonrails.org/mailman/listinfo/rails > > >
On 5/23/06, njmacinnes@gmail.com <njmacinnes@gmail.com> wrote:> Oh, ok. I even got beaten to the post... *red face*You mean that Dan already made a plugin (ActiveAttribute) like this? [see it here: http://lists.rubyonrails.org/pipermail/rails/2006-April/037180.html] Well I think the interesting part has yet to be coded. If ActiveAttribute would be able to detect changes between (1) the version of the model as specified in the model files and (2) the version of the model as specified in the database, it could do one of the following: - raise an exception, you need to upgrade (or downgrade!) the database first to continue, or - automatically upgrade and/or downgrade according to a configuration option. the upgrading and downgrading could be done using migrations -- this would need ActiveAttribute to hook up with migrations. Jon Smirl talks about such a solution here: http://lists.rubyonrails.org/pipermail/rails/2006-January/011395.html Point number 4 he makes in this post (yes, he number his points) is talking about (automatic) generation of migrations from model files. The main problem I can forsee is that generation of migrations out of the model files would never know the difference between: - adding and removing a column wiht an integer field, or - renaming a column with an integer field. There for some human judgement is needed, to solve this problem i would suggest to: 1. when the database is detected to be inconsistent with what is specified in the model files --> allways raise an exception 2. when generating the migrations to fix (either upgrade or downgrade) the database some occasional human input will be needed to know if a column is renamed, so some development data can be saved (this will only happen on development databases -- productiojn databases can then be modified by the generated migrations). Jon Smirl also mentions that the rails app should know which migration version it needs to run. From how i look at it this is not an issue. Lets say a group of developers is using svn to build a rails app then they are usually using migrations to communicate db schema changes already. When these migrations are generated from the model this does not changes the the situation, afaics.> On 23/05/06, njmacinnes@gmail.com <njmacinnes@gmail.com> wrote: > > I''ve thought of that before. Ok, so it''s never occured to me that up() > > and down() is against DRY (but you are right), I have wondered why I > > can''t just define the model along with the migrations myself.I hope that in this thead some design constraints can be sorted out, and some efford can be joined to make it into a rock solid solution.> > (the name of which I > > can''t remember and I''m not at my rails computer).ActiveRecord::Schema is (logically) already contained within the AR namespace: http://api.rubyonrails.com/classes/ActiveRecord/Schema.html> > I can''t see something like that making it into the rails framework > > because, as you say, there''s a lot of opposition, but that doesn''t > > bother me... a plugin is just as good.I don''t think there is a lot of opposition. Maybe some misconception. When someone sais "schem belongs in the db", he forgets that our lovely migrations and "active_record.schema_format = :ruby" allready put the schema out side of the db (namely in db/schema.rb (for reading), and in the migrations (for writing)). Though i shure hope that anyone with constructive opposition can voice up in this thread so we can all learn from it. Thanks, Cies Breijs. -- "Computer games don''t affect kids; I mean if Pac-Man affected us as kids, we''d all be running around in darkened rooms, munching magic pills and listening to repetitive electronic music." -- Kristian Wilson (Nintendo, Inc), 1989
Right, I''ve worked out how I''ll do it if I decide to. If not, I''m sure it''ll give someone else a few ideas. I''ll probs borrow some code from Dan''s plugin (with permission of course). Each model class (which will extend my new class) will have a two methods... an add method and a remove method. The add method contains any tables, fields and model validators/relationships that need adding to the current model migration. And the remove method, unsuprisingly removes them. Maybe an alter method too, but that could be difficult. The ''rake nathans_special_migrate'' command (I''ll have a think about what the actual command should be) would migrate the model through different versions, just like ''rake migrate'', but with the model instead. And, to make it absolutely DRY, to migrate to a lower version just reverses the methods (with a few modifications to make sure it''s extra dry). Any thoughts? Do people think they would like developing like this, or would you like it slightly different. Or do you think it''s just plain OTT. This is just a thought at the moment so it might not come into fruition... but if it does, it''ll probably incorporate suggestions here. Of course, I don''t have an international patent on the idea, so you''re quite welcome to race me (and probably win if you''re more than a novice (in fact, I''ll probably resign from competition if I hear anyone does)). -Nathan On 23/05/06, njmacinnes@gmail.com <njmacinnes@gmail.com> wrote:> I''ve thought of that before. Ok, so it''s never occured to me that up() > and down() is against DRY (but you are right), I have wondered why I > can''t just define the model along with the migrations myself. I guess > a plugin like that could be my project over the summer (I''m a student, > you see)... it sure would get me some good experience in Ruby and > rails. It would just mean having some uber-Daddy class which uses > methods from both ActiveRecord and the schema one (the name of which I > can''t remember and I''m not at my rails computer)... I really don''t > evision it being too hard at all. Of course, now that I''ve said this, > someone might beat me to it. > > I can''t see something like that making it into the rails framework > because, as you say, there''s a lot of opposition, but that doesn''t > bother me... a plugin is just as good. > > -N > > On 23/05/06, cies <cies.breijs@gmail.com> wrote: > > Dan, > > > > > I wrote a plugin that allows this syntax now: > > > > > > http://lists.rubyonrails.org/pipermail/rails/2006-April/037180.html > > > > > > It doesn''t deal with the migration side of the issue, just DRYing up the > > > validation rule specification. My plan was to have all the validation > > > rules in one place, with enough data to generate the migrations > > > from the model files later on. > > > > Nice! > > > > The syntax looks like a DRY-bliss to work with, i''m trying it right now. > > > > Quote of some of the plugins syntax: > > ---------------------- > > class Country < ActiveRecord::Base > > attribute :name, :string do |a| > > a.validates_length :within => 1..48 > > a.validates_format :with => /regexp/ > > a.validates_uniqueness > > end > > > > attribute :code_alpha2, :string do |a| > > a.validates_length :is => 2 > > a.validates_format :with => /\A[A-Z]+\z/ > > a.validates_uniqueness > > end > > > > # [...] > > end > > ---------------------- > > > > > > > > I totally lack the skills and experience to write a patch for this > > > > feature, I dont even know if this is a feature that the rails > > > > community can agree upon (when searching the web I found also quite > > > > some opposition towards ''putting the schema in the model''). > > > > > > It would be interesting for you to post links to the arguments against > > > the idea so that we may discuss them. > > > > I tried but I could not find it anymore. I remember that the main > > argument against was that "the schema belongs IN the database". In my > > inital post to this thread I also try to address this point by > > explaining that with migrations the schema is no longer ''only'' in the > > database. > > > > > > > I think we can all agree that we have to specify the schema someplace > > > at least once. > > > > Yeah, and most desireably: only once. No repeating of identifiers, > > symbols and mappings... please ;) > > > > > Traditionally rails has used the database as the single > > > reference point, and the models are generated from the current state of > > > the database tables. There has been talk of generating validation > > > rules from the table column definitions, and using foreign keys to > > > auto-generate the model relationships. The problem with this approach > > > is that every database has different capabilities, some have a really > > > rich column-level constraints and foreign keys, while others are > > > relatively > > > weak in this area like SQLite and older versions of MySQL. If you used > > > this approach and developed on one database you''d be forever tied > > > to that database. Rails is meant to be database agnostic, so it cannot > > > rely on the database as being the single reference point for much more > > > than just the column names, which leaves migrations or the model as > > > being > > > the only reliable place to define relationships and validation rules. > > > > Well the nice thing about Rails is that it can be db agnostic (though > > migrations), yet you also could use all features of your specific > > database and not use the migration. The developer has to make a > > trade-off between (in this case): > > - developing speed, portability, managability, and > > - database optimalisation > > > > Personally, for my projects I use the database as > > quick/dumb/structured storage. So no need for db optimalisations -- I > > happily use migrations to manage my db schema''s. > > > > > > > In Rails you have to specify everything twice: once in the migration > > > and once in the model. Effort has to be taken to keep them in sync, > > > so that the validates_length_of matches the :string column''s length > > > for example. You have to define configuration for the database, > > > and the configuration for the models -- with both needing to be in > > > agreement otherwise you''re risking introducing bugs. This continual > > > double configuration leads to a very un-DRY experience. > > > > Yep, I totally agree... Allthough I also understand the historics > > behind it: the dark days before migrations. > > > > > > > Given the choice of these two places to define relationships and > > > validation, I''d much rather have it in the model and generate the > > > migration from the current model. The model allows a much richer set > > > of constraints to be used than a database could ever hope to have. > > > I''m all for communicating a subset of those constraints to the > > > database when performing a migration -- send as much as the database > > > can understand, but the model should remain the single point > > > of reference. > > > > Here I differ slightly from your point of view. The developer should > > be able to choose whether he wants directly use the database, or that > > he likes to use highlevel stuff like migratons, your ActiveAttribute > > plugin or a future technique that hooks ActiveAttribute up with > > migrations. Rails should be flexible in this. > > Again: I, and probably many others using the db as dumb storage, would > > love to put all model stuff in the model files. > > > > > > > Thanks, > > Thank you Dan for the ActiveAttribute plugin! > > > > Cies Breijs. > > > > > > > > -- > > "Computer games don''t affect kids; I mean if Pac-Man affected us as > > kids, we''d all be running around in darkened rooms, munching magic > > pills and listening to repetitive electronic music." -- Kristian > > Wilson (Nintendo, Inc), 1989 > > _______________________________________________ > > Rails mailing list > > Rails@lists.rubyonrails.org > > http://lists.rubyonrails.org/mailman/listinfo/rails > > >
I made a little script to introspect the model files, this could be used to find all the "belongs_to", "attribute", etc, calls that should result in schema entries... -------- module ActiveRecord class Base def Base.method_missing(*args) puts "#{args[0].to_s} ( #{args[1..-1].join('', '')} )" end end end Dir["./app/models/*.rb"].sort.each do |f| puts "\n" + f require f end -------- It is just a little proof of concept -- it could be the basis of a rake task to ''pull the schema.rb straight out of the model files''. Please correct me if im doing any thing wrong! thanks, _cies. On 5/23/06, cies <cies.breijs@gmail.com> wrote:> Dan, > > > I wrote a plugin that allows this syntax now: > > > > http://lists.rubyonrails.org/pipermail/rails/2006-April/037180.html > > > > It doesn''t deal with the migration side of the issue, just DRYing up the > > validation rule specification. My plan was to have all the validation > > rules in one place, with enough data to generate the migrations > > from the model files later on. > > Nice! > > The syntax looks like a DRY-bliss to work with, i''m trying it right now. > > Quote of some of the plugins syntax: > ---------------------- > class Country < ActiveRecord::Base > attribute :name, :string do |a| > a.validates_length :within => 1..48 > a.validates_format :with => /regexp/ > a.validates_uniqueness > end > > attribute :code_alpha2, :string do |a| > a.validates_length :is => 2 > a.validates_format :with => /\A[A-Z]+\z/ > a.validates_uniqueness > end > > # [...] > end > ---------------------- > > > > > I totally lack the skills and experience to write a patch for this > > > feature, I dont even know if this is a feature that the rails > > > community can agree upon (when searching the web I found also quite > > > some opposition towards ''putting the schema in the model''). > > > > It would be interesting for you to post links to the arguments against > > the idea so that we may discuss them. > > I tried but I could not find it anymore. I remember that the main > argument against was that "the schema belongs IN the database". In my > inital post to this thread I also try to address this point by > explaining that with migrations the schema is no longer ''only'' in the > database. > > > > I think we can all agree that we have to specify the schema someplace > > at least once. > > Yeah, and most desireably: only once. No repeating of identifiers, > symbols and mappings... please ;) > > > Traditionally rails has used the database as the single > > reference point, and the models are generated from the current state of > > the database tables. There has been talk of generating validation > > rules from the table column definitions, and using foreign keys to > > auto-generate the model relationships. The problem with this approach > > is that every database has different capabilities, some have a really > > rich column-level constraints and foreign keys, while others are > > relatively > > weak in this area like SQLite and older versions of MySQL. If you used > > this approach and developed on one database you''d be forever tied > > to that database. Rails is meant to be database agnostic, so it cannot > > rely on the database as being the single reference point for much more > > than just the column names, which leaves migrations or the model as > > being > > the only reliable place to define relationships and validation rules. > > Well the nice thing about Rails is that it can be db agnostic (though > migrations), yet you also could use all features of your specific > database and not use the migration. The developer has to make a > trade-off between (in this case): > - developing speed, portability, managability, and > - database optimalisation > > Personally, for my projects I use the database as > quick/dumb/structured storage. So no need for db optimalisations -- I > happily use migrations to manage my db schema''s. > > > > In Rails you have to specify everything twice: once in the migration > > and once in the model. Effort has to be taken to keep them in sync, > > so that the validates_length_of matches the :string column''s length > > for example. You have to define configuration for the database, > > and the configuration for the models -- with both needing to be in > > agreement otherwise you''re risking introducing bugs. This continual > > double configuration leads to a very un-DRY experience. > > Yep, I totally agree... Allthough I also understand the historics > behind it: the dark days before migrations. > > > > Given the choice of these two places to define relationships and > > validation, I''d much rather have it in the model and generate the > > migration from the current model. The model allows a much richer set > > of constraints to be used than a database could ever hope to have. > > I''m all for communicating a subset of those constraints to the > > database when performing a migration -- send as much as the database > > can understand, but the model should remain the single point > > of reference. > > Here I differ slightly from your point of view. The developer should > be able to choose whether he wants directly use the database, or that > he likes to use highlevel stuff like migratons, your ActiveAttribute > plugin or a future technique that hooks ActiveAttribute up with > migrations. Rails should be flexible in this. > Again: I, and probably many others using the db as dumb storage, would > love to put all model stuff in the model files. > > > > Thanks, > Thank you Dan for the ActiveAttribute plugin! > > Cies Breijs. > > > > -- > "Computer games don''t affect kids; I mean if Pac-Man affected us as > kids, we''d all be running around in darkened rooms, munching magic > pills and listening to repetitive electronic music." -- Kristian > Wilson (Nintendo, Inc), 1989 >-- "Computer games don''t affect kids; I mean if Pac-Man affected us as kids, we''d all be running around in darkened rooms, munching magic pills and listening to repetitive electronic music." -- Kristian Wilson (Nintendo, Inc), 1989
>"Why is the model specified in two separate places?"I may be misunderstanding you, and I''m probably being too pedantic (it''s an accusation that is often made), but is the model _really_ being specified in two seperate places? From the perspective of your application, the model is distributed between the database for the schema definition, and your model file for validation and business logic. As far as I understand it, schema.rb and your migrations are tools which allow changes to be applied to the schema without having to work directly with the underlying database - and that''s all. Once you''re working with the application schema.rb and migrations don''t really come into play. Aside from repeating myself between self.up and self.down within a single migration I don''t find defining the schema in a migration a violation of DRY. Could you maybe elaborate on your reasons for wanting to specify the schema in the model? cheers, Ben
On 5/24/06, Ben and Kaz Askins <ror.teamaskins@gmail.com> wrote:> >"Why is the model specified in two separate places?" > > I may be misunderstanding you, and I''m probably being too pedantic > (it''s an accusation that is often made), but is the model _really_ > being specified in two seperate places? From the perspective of your > application, the model is distributed between the database for the > schema definition, and your model file for validation and business > logic. As far as I understand it, schema.rb and your migrations are > tools which allow changes to be applied to the schema without having > to work directly with the underlying database - and that''s all. Once > you''re working with the application schema.rb and migrations don''t > really come into play.once you are making a table called ''books'' and a model file called ''book.rb'' containing a class called ''Book'' you are repeating yourself. When you make a "belongs_to :category" and a column ''category_id'' on the ''books'' you are repeating your self again. One more: if you make a "validates_presence_of :title" you need a ''title'' column in the corresponding table (okay this was a weak one). Then consider these: --------------------- class Author < ActiveRecord::Base has_many :posts attribute :name, :string attribute :email, :string do |a| a.validate_format :with => /regex/ end end --------------------- class Post < ActiveRecord::Base belongs_to :author # automatically adds author_id to the table attribute :title, :string do |a| a.validate_presence a.validate_length :in => 3..100 end attribute :summary, :text attribute :content, :text end --------------------- [from: http://lists.rubyonrails.org/pipermail/rails/2006-January/011325.html] Now i forsee that a (not yet existing) rake task could, at any time, turn these model files into: - a db/schema_from_the_model.rb file - the needed migrations from a given state of the db to this schema_from_the_model And an (optinal) exception is raised when the database on the model files are our of sync (so you know when you forgot to run the previously mentioned rake task). IMO this would keep the model ''together'' better specified. Yet (as for migrations_) you dont HAVE to use it, you could always use it the old way.> Aside from repeating myself between self.up and self.down within a > single migration I don''t find defining the schema in a migration a > violation of DRY.As i just pointed out, it is -- maybe not a violation but a -- minor offence to DRY. Yet there is also the effect of keeping things at one place only.> Could you maybe elaborate on your reasons for wanting to specify the > schema in the model?I feel you are asking me to repeat my self (lol)... But since im passionate about it, here I go (please note that this only hold true for people already using migration for modifying the db schema): Why should it be possible to put the schema in the model? 1. we should not spread our schema over many places (relations are in the model file, changes in the migrations, the database contains the db specific version and db/schema.rb the agnostic). 2. it makes is easier to work with: no switching, directly seeing the attributes of a model, and in an ideal situation changing the schema in the model file should propagate fairly easy (through migrations) to the db. 3. we could do database specifications based on the validations as pointed out by Dan here: http://lists.rubyonrails.org/pipermail/rails/2006-January/012133.html 4. you are repeating yourself less (you say not so much less, okay, i understand that), by not repeating the up()-down() thing in the migrations, but also autocreation of a ''author_id'' column in case "belongs_to :author" is specified, and not repeating column and table names. 5. since we''re going an abstraction level higher new (possibly not yet imagined) stuff will become possible -- let me think, uuhhh, yes: maybe an list of options can be implemented easily like: -------------- Account < ActiveRecord::Base attribute :status, :options => [''pending'', ''active'', ''disabled'', ''error''] # ... end -------------- The way the status attribute can be implemented in a way that is possible using the underlying db, the ''status'' attribute can the automatically be validated to be one of the options, and it can supply a method :status_options to return the array. Well i cane to 5 reasons for why to specify the schema in the model... Maybe there are more, but im going to bed now ;) Ben, I hope I managed to enthouse you a tiny bit (or more) for my case! Thanks for asking me this, it helped me generating a list of 5 advantages. Cies Breijs.
So i converted Dan''s ActiveAttribute plugin into a plugin names "acts_as_schema", and made a few modifications. For what i see some things still need to happen: - belongs_to''s must be noticed by the plugin so that can result in a column for the schema - somehow this plugin should collect all the specified attributes and belong_to''s that are specified so it can build the schema when its asked to - i have not figured out yet when the check between the model specified in the database and the model file should occur and when to trigger it maybe this is the whole wrong approach, maybe it should not be done from a plugin but from an external tool run by a rake task or so. the code: __________________________- module ActiveRecord #:nodoc: module Acts #:nodoc: # This is a very early shot at making a ''acts_as_schema'' extension that should # enable one to use the model as (or to generate) a schema... # # Right now it is basically a port of Dan Kubb''s ActiveAttribute to a ''acts_as_*'' plugin. # http://onautopilot.com/oss/rails/active_attribute.tgz # # # Example: # # class Country < ActiveRecord::Base # acts_as_schema # # attribute :name, :string do |a| # a.validates_length :within => 1..48 # a.validates_format :with => /regex/ # a.validates_uniqueness # end # # attribute :description, :string do |a| # a.validates_length :within => 1..999 # end # # attribute :phone_code, :integer # # attribute :minutes, :integer # end # module Schema def self.included(base) # :nodoc: base.extend ClassMethods end module ClassMethods def acts_as_schema # don''t allow multiple calls return if self.included_modules.include?(ActiveRecord::Acts::Schema::ActMethods) # class << self # alias_method :internal_belongs_to, :belongs_to # end send :include, ActiveRecord::Acts::Schema::ActMethods end end module ActMethods def self.included(base) # :nodoc: base.extend ClassMethods end module ClassMethods def attribute(name, type, options = {}) configuration = { :allow_nil => true }.update(options) unless configuration[:allow_nil] # XXX: get around rails bug: http://dev.rubyonrails.org/ticket/3334 if type == :boolean configuration = { :message => ActiveRecord::Errors.default_error_messages[:blank], :on => :save } # can''t use validates_each here, because it cannot cope with nonexistent attributes, # while errors.add_on_empty can send(validation_method(configuration[:on])) do |record| unless configuration[:if] and not evaluate_condition(configuration[:if], record) value = record.respond_to?(name.to_s) ? record.send(name.to_s) : record[name.to_s] record.errors.add(attr_name, configuration[:message]) if value.nil? end end else validates_presence_of name end end attribute ActiveRecord::Acts::Schema::Attribute.new(self, name, type, configuration) yield attribute if block_given? attribute end end end class Attribute def initialize(model, name, type, default_options = {}) @model, @name, @type, @default_options = model, name, type, default_options Attribute.add(model, name, type, default_options) end def validates_length(options = {}) raise ''cannot validate the length of an integer'' if @type == :integer configuration = @default_options.clone unless options.has_key?(:is) || options.has_key?(:within) || options.has_key?(:in) configuration.merge! case @type when :string : { :maximum => 255 } when :text : { :maximum => 65535 } else {} end end @model.validates_length_of @name, configuration.merge(options) end def validates_numericality(options = {}) configuration = case @type when :integer : @default_options.merge(:only_integer => true) when :float : @default_options.merge(:only_integer => false) else @default_options end @model.validates_numericality_of @name, configuration.merge(options) end def method_missing(method, options = {}, &block) if method.to_s =~ /\A(validates_[a-z]+(?:_[a-z]+)*)\z/ case when @model.respond_to?($1 + ''_of'') : @model.send($1 + ''_of'', @name, @default_options.merge(options), &block) when @model.respond_to?($1) : @model.send($1, @name, @default_options.merge(options), &block) end end || super end # i have no idea if what im doing next does even make remote sense... # i hope to accomplish to somehow ''know'' all arrtibutes that are specified, # so they can be checked with the @attributes.keys or with the schema def Attribute.schema_hash @@attributes_for_schema or ''<nothing>'' end def Attribute.add(model, name, type, default_options = {}) @@attributes_for_schema ||= {} @@attributes_for_schema[model] ||= {} @@attributes_for_schema[model][name] = [type, default_options] end end end end end ActiveRecord::Base.send :include, ActiveRecord::Acts::Schema