Just ran into something where a gem we were using was overriding a method ActiveRecord that was the Rails 3.1/3.2 version of the method instead of the Rails 4 version. Its tests would not have noticed any error after changing the AR dependency to 4.0 because it was only testing the modified method, not running all of the AR tests for the affected module. The first response to this might be: "So what? People have been monkey patching Rails for what seems like forever, and they need to diff/read release notes, and be responsible for their own #$%t." Sure, but really- how many people are going to do that? i.e. http://www.youtube.com/watch?v=8cT_Ulmcrys It is realistic to think that maybe (like in Rails 5), the tests could be extracted for each component somehow to make it easier to run tests for one or more modules (or an entire gem), possibly allowing the ability to easily exclude specific tests. It might even be as easy as just another gemspec in the same directory with perhaps some gem metadata to indicate what the method is to run the tests (which might get a little crazy dependency-wise if you''ve got different test framework version dependencies, etc., but I doubt it is that much crazier than how things are currently). I''m sure someone has already asked and/or gone down this road before/thought about it, but thought I''d bring it up perhaps for further discussion. This isn''t so much a Rails problem as it is something that Rails could do to really help its community. Maybe it could even be done prior to a major version, since it wouldn''t affect Rails that much at all, functionally (I''d hope). Thanks! -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-core+unsubscribe@googlegroups.com. To post to this group, send email to rubyonrails-core@googlegroups.com. Visit this group at http://groups.google.com/group/rubyonrails-core. For more options, visit https://groups.google.com/groups/opt_out.
> It might even be as...And to head of complaints about that statement- I mean that possibly one of us could write a gem that would make running tests from a "modularized tests" gem easier and more standardized. -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-core+unsubscribe@googlegroups.com. To post to this group, send email to rubyonrails-core@googlegroups.com. Visit this group at http://groups.google.com/group/rubyonrails-core. For more options, visit https://groups.google.com/groups/opt_out.
I''m a bit confused as to what you''re proposing. You can run tests for any of the sub-gems by cd-ing into each gem, and running the tests, you can choose which ones to run by doing all the usual minitest stuff. -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-core+unsubscribe@googlegroups.com. To post to this group, send email to rubyonrails-core@googlegroups.com. Visit this group at http://groups.google.com/group/rubyonrails-core. For more options, visit https://groups.google.com/groups/opt_out.
Problem: If I extend ActiveRecord in such a way that it would break ActiveRecord, I have no way of *easily* running ActiveRecord''s own tests on my modified version of ActiveRecord. TravisCI, etc. do not help in this regard because if I only run my tests against the modified method (in a BDD or TDD scenario), those tests may continue to pass. Requirement: I want to extend ActiveRecord by patching it, but I want to ensure that when ActiveRecord changes, all I have to do is to use the new ActiveRecord dependency and within my application or my gem I can run the ActiveRecord tests (or tests for a part of ActiveRecord) in my application or gem environment. Proposed Solution Options: #1 I just copy ActiveRecord''s tests into my gem or application and modify them as needed/setup the environment so that those tests will run. Whenever I update ActiveRecord, I have to repeat this exercise. This seems like a lot of work, so as an app/gem author I avoid it. Because I avoid it, I run into the problem again later. People that use my app/gem see weird bugs. They think these are issues with Rails. They post questions to S.O. and various groups and issues to GitHub, wasting everyone''s time. Eventually the bug is fixed. Repeat. Everyone becomes unhappy. This unhappiness spreads and manifests itself in decline in health and moral corruption. This continues until it is replaced with a "new world order" (NWO). The NWO has supreme power, leading to corruption. Terror reigns. World ends. #2 Rails would provide gems that were packages of tests that could be run against an existing loaded (and possibly modified in initializer, etc.) version of ActiveRecord. By depend on a gem that contains ActiveRecord tests, they could selectively run the tests they are interested in (or sets of them, or all of them). Bugs are significantly reduced in gems that monkey patch Rails. Ruby developers become wealthy and happy. They collectively organize their funds to build a large spaceship in the shape of a Ruby and they all fly to a distant galaxy and start a Utopia. Earth is now Ruby-free. Terror reigns. World ends. Currently the ActiveRecord gem does not include the tests, from what I see in my local dir and in its gemspec. I''m proposing something like an ''activerecord-tests'' gem, and similar for each major Rails project. On Thursday, August 8, 2013 9:40:02 AM UTC-4, Steve Klabnik wrote:> > I''m a bit confused as to what you''re proposing. You can run tests for > any of the sub-gems by cd-ing into each gem, and running the tests, > you can choose which ones to run by doing all the usual minitest > stuff. >-- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-core+unsubscribe@googlegroups.com. To post to this group, send email to rubyonrails-core@googlegroups.com. Visit this group at http://groups.google.com/group/rubyonrails-core. For more options, visit https://groups.google.com/groups/opt_out.
I''m not sure that encouraging people to extend activerecord in this way is a good idea. If there''s something that needs extension that doesn''t have a public API, it should be added, not encouraging monkey-patching. -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-core+unsubscribe@googlegroups.com. To post to this group, send email to rubyonrails-core@googlegroups.com. Visit this group at http://groups.google.com/group/rubyonrails-core. For more options, visit https://groups.google.com/groups/opt_out.
> If there''s something that needs extension that doesn''t have a public API,it should be added, not encouraging monkey-patching. I can understand why Rails would not want to encourage monkey-patching, but I also don''t see why it could not decouple its tests so that they could be run against any (or a compatible) loaded version of the rails project. Rails is for modularity (to the extent that it helps rather than hurts), so why not provide sets of tests for each component that could test that the component is compatible with Rails expected functionality? After the work is done to decouple tests from each of the Rails projects, it should be easy to maintain such a setup because Rails itself could test its gems with its test gems. I disagree with the following though: "If there''s something that needs extension that doesn''t have a public API, it should be added". From what I have read from DHH and others, the general thought is that Rails/Ruby is playdoh, not legos. Turning everything that needs to be extended into a method, etc. seems fine at first in-theory, and in the case that I mention it would have helped; specifically it would help if the existing_records variable in the assign_nested_attributes_for_collection_association method in NestedAttributes had always been a method. However, how far can you really take that? Rails is never going to be everything to everyone, so there are going to be times when it is patched like this. Why not embrace it enough to offer an easier way to certify that something "works in the way Rails expects" without causing undue hardship on the Rails dev team? Thanks for considering this. On Thursday, August 8, 2013 3:35:52 PM UTC-4, Steve Klabnik wrote:> > I''m not sure that encouraging people to extend activerecord in this > way is a good idea. If there''s something that needs extension that > doesn''t have a public API, it should be added, not encouraging > monkey-patching. >-- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-core+unsubscribe@googlegroups.com. To post to this group, send email to rubyonrails-core@googlegroups.com. Visit this group at http://groups.google.com/group/rubyonrails-core. For more options, visit https://groups.google.com/groups/opt_out.
What Steve said, plus I don''t think shipping this as a Gem would work
as
nicely as you imagine – for one thing, if you are doing something like this
you''d likely have to target and test for multiple version of
Rails/AR/...
using a gem for this does not allow you to switch easily between versions.
My recommendation would depend on the specific use case you have in mind:
If you are fixing a bug, then...
1. Fork Rails
2. Fix the bug (and submit a pull request)
3. Edit your Gem file to use your fork
4. (Wait for your PR to be merged and the next release to come out, then
change it back)
If you are adding a feature that depends on documented APIs, then...
1. In your tests, mock out the documented behaviour according to the
documentation and trust that Rails won''t break these contracts
unexpectedly
2. In your gemspec, be explicit about the versions you support (tested)
– i.e. add_dependency(''activerecord'', ''>=
3.1.0'', ''< 4.0.0'')
3. When a new "minor" release (4.1.0) came out, make sure the API
hasn''t
changed, run the tests, then change your gemspec
– add_dependency(''activerecord'', ''>=
3.1.0'', ''< 4.1.0'')
If you are relying on undocumented APIs, then...
1. Well, know that you''re in for quite a ride and you''re
fighting a lost
battle...
2. Clone rails into /vendor/rails as a Git submodule, and remember
EXCLUDE it in your gemspec
3. Write a rake task (e.g. rake test:vendor) that runs that tests in the
vendorized rails in ways that fits your need
4. Write another rake task (rake vendor:rails version) that does
essentially "cd vendor/rails && git checkout
tags/#{version}"
5. Write another rake task (e.g. rake test:vendor:all) that iterates
through all the versions you care about (rake vendor:rails 3.1.0 &&
rake
test:vendor && rake vendor:rails ...)
6. Cross your fingers
On Thu, Aug 8, 2013 at 12:35 PM, Steve Klabnik
<steve@steveklabnik.com>wrote:
> I''m not sure that encouraging people to extend activerecord in
this
> way is a good idea. If there''s something that needs extension that
> doesn''t have a public API, it should be added, not encouraging
> monkey-patching.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Ruby on Rails: Core" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to rubyonrails-core+unsubscribe@googlegroups.com.
> To post to this group, send email to rubyonrails-core@googlegroups.com.
> Visit this group at http://groups.google.com/group/rubyonrails-core.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>
--
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Core" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to rubyonrails-core+unsubscribe@googlegroups.com.
To post to this group, send email to rubyonrails-core@googlegroups.com.
Visit this group at http://groups.google.com/group/rubyonrails-core.
For more options, visit https://groups.google.com/groups/opt_out.
Thanks for the details set of steps for each scenario. The third one is basically the one in this case, and as you can see from reading the steps you wrote- no one is typically going to go through that trouble. One thing I can say for certain is that if everyone that has every contributed to rails that also has monkey-patched some part of Rails rose their hands, most would be raising their hands. I don''t care enough about it to be the only voice in favor though. On Thursday, August 8, 2013 5:02:39 PM UTC-4, Godfrey Chan wrote:> > What Steve said, plus I don''t think shipping this as a Gem would work as > nicely as you imagine – for one thing, if you are doing something like this > you''d likely have to target and test for multiple version of Rails/AR/... > using a gem for this does not allow you to switch easily between versions. > > My recommendation would depend on the specific use case you have in mind: > > If you are fixing a bug, then... > > > 1. Fork Rails > 2. Fix the bug (and submit a pull request) > 3. Edit your Gem file to use your fork > 4. (Wait for your PR to be merged and the next release to come out, > then change it back) > > If you are adding a feature that depends on documented APIs, then... > > 1. In your tests, mock out the documented behaviour according to the > documentation and trust that Rails won''t break these contracts unexpectedly > 2. In your gemspec, be explicit about the versions you support > (tested) – i.e. add_dependency(''activerecord'', ''>= 3.1.0'', ''< 4.0.0'') > 3. When a new "minor" release (4.1.0) came out, make sure the API > hasn''t changed, run the tests, then change your gemspec > – add_dependency(''activerecord'', ''>= 3.1.0'', ''< 4.1.0'') > > If you are relying on undocumented APIs, then... > > 1. Well, know that you''re in for quite a ride and you''re fighting a > lost battle... > 2. Clone rails into /vendor/rails as a Git submodule, and remember > EXCLUDE it in your gemspec > 3. Write a rake task (e.g. rake test:vendor) that runs that tests in > the vendorized rails in ways that fits your need > 4. Write another rake task (rake vendor:rails version) that does > essentially "cd vendor/rails && git checkout tags/#{version}" > 5. Write another rake task (e.g. rake test:vendor:all) that iterates > through all the versions you care about (rake vendor:rails 3.1.0 && rake > test:vendor && rake vendor:rails ...) > 6. Cross your fingers > > > > > > > > > On Thu, Aug 8, 2013 at 12:35 PM, Steve Klabnik <st...@steveklabnik.com<javascript:> > > wrote: > >> I''m not sure that encouraging people to extend activerecord in this >> way is a good idea. If there''s something that needs extension that >> doesn''t have a public API, it should be added, not encouraging >> monkey-patching. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Ruby on Rails: Core" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to rubyonrails-co...@googlegroups.com <javascript:>. >> To post to this group, send email to rubyonra...@googlegroups.com<javascript:> >> . >> Visit this group at http://groups.google.com/group/rubyonrails-core. >> For more options, visit https://groups.google.com/groups/opt_out. >> >> >> >-- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-core+unsubscribe@googlegroups.com. To post to this group, send email to rubyonrails-core@googlegroups.com. Visit this group at http://groups.google.com/group/rubyonrails-core. For more options, visit https://groups.google.com/groups/opt_out.
Also- On Thursday, August 8, 2013 5:02:39 PM UTC-4, Godfrey Chan wrote:> What Steve said, plus I don''t think shipping this as a Gem would work as > nicely as you imagine – for one thing, if you are doing something like this > you''d likely have to target and test for multiple version of Rails/AR/... > using a gem for this does not allow you to switch easily between versions. >That is already the case with gems. Each gem is a different version, so each test gem would have a corresponding version. I can use the appraisal gem to test in multiple versions of rails, or just activerecord, etc. So, for tests, I''d just pair the version of activerecord with the gem corresponding to its tests. Eventually if others started to use related conventions, you might not even have to specify the test gem- it would just go and get it and test it just because you loaded that gem during your tests. Whenever a version of AR was cut, its tests would be cut as their gem. The test gem would contain the dummy rails project if needed, etc. In this way, you could even find out what parts (tests) of the API would fail from an earlier version of Rails, etc. It opens up a lot of interesting possibilities, other than just testing monkey-patched code. -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-core+unsubscribe@googlegroups.com. To post to this group, send email to rubyonrails-core@googlegroups.com. Visit this group at http://groups.google.com/group/rubyonrails-core. For more options, visit https://groups.google.com/groups/opt_out.
At the very least it would be nice if the tests were included in the Rails gems: http://docs.rubygems.org/read/chapter/20#test_files That way you wouldn''t have to go to github to get the tests. Why aren''t tests currently included in the gem? On Thursday, August 8, 2013 5:33:04 PM UTC-4, Gary Weaver wrote:> > Also- > > On Thursday, August 8, 2013 5:02:39 PM UTC-4, Godfrey Chan wrote: > >> What Steve said, plus I don''t think shipping this as a Gem would work as >> nicely as you imagine – for one thing, if you are doing something like this >> you''d likely have to target and test for multiple version of Rails/AR/... >> using a gem for this does not allow you to switch easily between versions. >> > > That is already the case with gems. Each gem is a different version, so > each test gem would have a corresponding version. I can use the appraisal > gem to test in multiple versions of rails, or just activerecord, etc. So, > for tests, I''d just pair the version of activerecord with the gem > corresponding to its tests. Eventually if others started to use related > conventions, you might not even have to specify the test gem- it would just > go and get it and test it just because you loaded that gem during your > tests. > > Whenever a version of AR was cut, its tests would be cut as their gem. The > test gem would contain the dummy rails project if needed, etc. In this way, > you could even find out what parts (tests) of the API would fail from an > earlier version of Rails, etc. It opens up a lot of interesting > possibilities, other than just testing monkey-patched code. >-- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-core+unsubscribe@googlegroups.com. To post to this group, send email to rubyonrails-core@googlegroups.com. Visit this group at http://groups.google.com/group/rubyonrails-core. For more options, visit https://groups.google.com/groups/opt_out.
> That way you wouldn''t have to go to github to get the tests. Why aren''t > tests currently included in the gem? > > So your current workflow is* Download the github://.. activerecord gem in your Gemfile * Check out the version (e.g. 4.0)? * Inject your changes * Run rake in the activerecord/ dir? And, instead, something like that would be cool? * Link the activerecord-tests in your Gemfile along with version * In your gem test, do something like ActiveRecord::TestSuite.run ? I like it!> > On Thursday, August 8, 2013 5:33:04 PM UTC-4, Gary Weaver wrote: >> >> Also- >> >> On Thursday, August 8, 2013 5:02:39 PM UTC-4, Godfrey Chan wrote: >> >>> What Steve said, plus I don''t think shipping this as a Gem would work as >>> nicely as you imagine – for one thing, if you are doing something like this >>> you''d likely have to target and test for multiple version of Rails/AR/... >>> using a gem for this does not allow you to switch easily between versions. >>> >> >> That is already the case with gems. Each gem is a different version, so >> each test gem would have a corresponding version. I can use the appraisal >> gem to test in multiple versions of rails, or just activerecord, etc. So, >> for tests, I''d just pair the version of activerecord with the gem >> corresponding to its tests. Eventually if others started to use related >> conventions, you might not even have to specify the test gem- it would just >> go and get it and test it just because you loaded that gem during your >> tests. >> >> Whenever a version of AR was cut, its tests would be cut as their gem. >> The test gem would contain the dummy rails project if needed, etc. In this >> way, you could even find out what parts (tests) of the API would fail from >> an earlier version of Rails, etc. It opens up a lot of interesting >> possibilities, other than just testing monkey-patched code. >> >-- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-core+unsubscribe@googlegroups.com. To post to this group, send email to rubyonrails-core@googlegroups.com. Visit this group at http://groups.google.com/group/rubyonrails-core. For more options, visit https://groups.google.com/groups/opt_out.
> > * Download the github://.. activerecord gem in your Gemfile >Sorry, that should read "Download the activerecord gem" :-) -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-core+unsubscribe@googlegroups.com. To post to this group, send email to rubyonrails-core@googlegroups.com. Visit this group at http://groups.google.com/group/rubyonrails-core. For more options, visit https://groups.google.com/groups/opt_out.
On Thursday, August 8, 2013 8:43:14 PM UTC-4, Nick Sutterer wrote:> > And, instead, something like that would be cool? > > * Link the activerecord-tests in your Gemfile along with version > * In your gem test, do something like ActiveRecord::TestSuite.run > > ? > > I like it! >Exactly. But that''s not all! What if that single line called/required code that would iterate Gem.loaded_specs, find gems that have metadata indicating that they have tests for another gem, and if those gems are loaded, it runs those tests? This would be for the long-running CI builds, and would basically ensure that every library you had (that followed this testing practice) was fully vetted. That way you wouldn''t have to worry about someone accidentally putting a breaking change in a patch version that you didn''t notice because you didn''t run some other dependencies tests that required it? -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-core+unsubscribe@googlegroups.com. To post to this group, send email to rubyonrails-core@googlegroups.com. Visit this group at http://groups.google.com/group/rubyonrails-core. For more options, visit https://groups.google.com/groups/opt_out.
On Thursday, August 8, 2013 10:18:45 PM UTC-4, Gary Weaver wrote:> > What if that single line called/required code that would iterate > Gem.loaded_specs, find gems that have metadata indicating that they have > tests for another gem, and if those gems are loaded, it runs those tests? > This would be for the long-running CI builds, and would basically ensure > that every library you had (that followed this testing practice) was fully > vetted. That way you wouldn''t have to worry about someone accidentally > putting a breaking change in a patch version that you didn''t notice because > you didn''t run some other dependencies tests that required it? >As a disclaimer, this is vaguely similar to what the rubygems-test gem did, but it just tested or offered to test on *gem install* (auto_test_on_install and test_on_install). Since test.rubygems.org no longer responds, I guess testing on install didn''t pan out for them. Testing on demand is more useful. Thinking about this more, I think that something like convention that you mentioned, Nick, might be a good way to start to pave the road to later having a first-class attribute in the gemspec. Right now the gem versioning convention is ::VERSION at the end of the module "namespace" for the gem, e.g. in the activerecord-foobar gem, we''d use ActiveRecord::Foobar::VERSION and set that as the gemspec version for the gem. So to indicate the primary runner for a modular testing gem, maybe for the activerecord-foobar case, it would use: ActiveRecord::Foobar::TESTS or similar, that way even before it caught on and there was a ''default_test_executable'' in gemspec, we could assume the convention that ActiveRecord::Foobar::TESTS would be executed as a Ruby script that would run the tests, regardless of framework required. We could also drop the caps convention and just use Tests. Eventually, maybe you could similarly have Tests for each module as a known convention (e.g. ActiveRecord::NestedAttributes::Tests) with a standard way regardless of the test framework to execute them, though in the beginning, I think it would be fine to just to even just have some way to run single/multiple tests from ActiveRecord in a different environment at all in a way that would reduce conflicts. Doing this would take some reorganizing and modifications in Rails. Simply having models in a "models" directory and no namespacing for the tests as you can see in: https://github.com/rails/rails/blob/8f37ba81abbd13b935ce0f26574ff2b720b552f1/activerecord/lib/active_record/nested_attributes.rb wouldn''t work as well, but add an activerecord directory to contain the models directory and a module ActiveRecord::NestedAttributes::Tests or similar towards the top of the test, and you''re part of the way there. Rails would benefit from the extra eyes and executions of its tests, so it might have to be approached from a task like "clean up tests in such a way that there is a standard for how you would test a particular module (and even a particular method), even from another gem, so that the chance of conflicts is reduced" and let the rest fall out from that. In the end, I really don''t care how it is done, as long as there is a feasible way to do it that doesn''t involve a lot of time consuming steps. What quality project would want to make it difficult to run tests? -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-core+unsubscribe@googlegroups.com. To post to this group, send email to rubyonrails-core@googlegroups.com. Visit this group at http://groups.google.com/group/rubyonrails-core. For more options, visit https://groups.google.com/groups/opt_out.
On Friday, August 9, 2013 7:47:06 AM UTC-4, Gary Weaver wrote:> > as you can see in: > https://github.com/rails/rails/blob/8f37ba81abbd13b935ce0f26574ff2b720b552f1/activerecord/lib/active_record/nested_attributes.rb >or rather: https://github.com/rails/rails/blob/8f37ba81abbd13b935ce0f26574ff2b720b552f1/activerecord/test/cases/nested_attributes_test.rb -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-core+unsubscribe@googlegroups.com. To post to this group, send email to rubyonrails-core@googlegroups.com. Visit this group at http://groups.google.com/group/rubyonrails-core. For more options, visit https://groups.google.com/groups/opt_out.