With Rails, is "rake spec" supposed to automatically run under the test environment? I have to do "rake spec RAILS_ENV=test" in order to get the environments/test.rb file to load. Also, are the standard Test::Unit fixtures automatically loaded before each test in each spec file? This appears to be the case based on my current spec''s behavior, but wanted to verify. Thanks, Wes
In your spec/spec_helper.rb file you should see a line at the to which says RAILS_ENV="test". If you haven''t modified your spec_helper.rb file I would remove it and re-run "script/generate rspec". It should be there. I just generated a new Rails project with rspec and rspec-rails gems. Here''s the top of my spec/spec_helper.rb: # This file is copied to ~/spec when you run ''ruby script/generate rspec'' # from the project root directory. ENV["RAILS_ENV"] = "test" require File.expand_path(File.dirname(__FILE__) + "/../config/environment") require ''spec'' require ''spec/rails'' Zach On Wed, Oct 8, 2008 at 3:15 PM, Wes Gamble <weyus at att.net> wrote:> With Rails, is "rake spec" supposed to automatically run under the test > environment? I have to do "rake spec RAILS_ENV=test" in order to get the > environments/test.rb file to load. > > Also, are the standard Test::Unit fixtures automatically loaded before each > test in each spec file? This appears to be the case based on my current > spec''s behavior, but wanted to verify. > > Thanks, > Wes > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users >-- Zach Dennis http://www.continuousthinking.com http://www.mutuallyhuman.com
Zach, Zach Dennis wrote:> # This file is copied to ~/spec when you run ''ruby script/generate rspec'' > # from the project root directory. > ENV["RAILS_ENV"] = "test" > require File.expand_path(File.dirname(__FILE__) + "/../config/environment") > require ''spec'' > require ''spec/rails'' >This is exactly what the top of my spec_helper.rb file looks like. I have several specs, and the success of running them is different depending on what I do. I''m really just trying to figure out how to take advantage of the "rake spec" task. I''m on Windows XP. Rails 2.1.0. Rspec and Rspec-Rails 1.1.8 gems. No Rspec plugins. Freshly generate rspec stuff using "ruby script/generate rspec" 1) If I run each of my specs separately on the command line using "spec spec/whatever.rb" they run fine. 2) If I run "spec spec/*spec.rb", then all _but one_ run without error (note that the one that files can be run successfully using a direct call to it as in #1 above). 3) If I run "rake spec", I get multiple failures, all because the config/environments/test.rb file is not loaded (I know this because they fail due to the lack of a variable that is set only in test.rb). The spec that failed in #2 works in this case. 4) If I run "rake spec RAILS_ENV=test", all _but one_ run without error (same error as in #2). I draw the following conclustions: 1) I only get the color output if I use "rake spec" (not sure why) 2) spec loads test.rb by default, but "rake spec" does not and that''s why I have to pass RAILS_ENV=test to the "rake" command. Wes rspec 1.1.8 gem rspec-rails 1.1.8 gem
Correction to above: Wes Gamble wrote:> 2) If I run "spec spec/*spec.rb", then all _but one_ run without error > (note that the one that fails can be run successfully using a direct > call to it as in #1 above).-- Posted via http://www.ruby-forum.com/.
On Wed, Oct 8, 2008 at 3:41 PM, Wes Gamble <weyus at att.net> wrote:> Zach, > > Zach Dennis wrote: >> >> # This file is copied to ~/spec when you run ''ruby script/generate rspec'' >> # from the project root directory. >> ENV["RAILS_ENV"] = "test" >> require File.expand_path(File.dirname(__FILE__) + >> "/../config/environment") >> require ''spec'' >> require ''spec/rails'' >> > > This is exactly what the top of my spec_helper.rb file looks like. > > I have several specs, and the success of running them is different depending > on what I do. I''m really just trying to figure out how to take advantage of > the "rake spec" task. > I''m on Windows XP. Rails 2.1.0. Rspec and Rspec-Rails 1.1.8 gems. No > Rspec plugins. Freshly generate rspec stuff using "ruby script/generate > rspec" > > 1) If I run each of my specs separately on the command line using "spec > spec/whatever.rb" they run fine. > > 2) If I run "spec spec/*spec.rb", then all _but one_ run without error (note > that the one that files can be run successfully using a direct call to it as > in #1 above). > > 3) If I run "rake spec", I get multiple failures, all because the > config/environments/test.rb file is not loaded (I know this because they > fail due to the lack of a variable that is set only in test.rb). The spec > that failed in #2 works in this case. > > 4) If I run "rake spec RAILS_ENV=test", all _but one_ run without error > (same error as in #2). > > I draw the following conclustions: > > 1) I only get the color output if I use "rake spec" (not sure why) > > 2) spec loads test.rb by default, but "rake spec" does not and that''s why I > have to pass RAILS_ENV=test to the "rake" command.What is test.rb? As for getting different results if you run files individually, with the spec command or with the rake command, that is *usually* a sign that there is some state leaking between examples. ''rake spec'' and ''spec spec'' don''t load the files in the same order, so the fact that you also get different failures from each of those also supports this theory in your case. Do you have a spec/spec.opts file? What does that look like?> Wes > > rspec 1.1.8 gem > rspec-rails 1.1.8 gem
David Chelimsky wrote:> On Wed, Oct 8, 2008 at 3:41 PM, Wes Gamble <weyus at att.net> wrote: > >> Zach, >> >> Zach Dennis wrote: >> >>> # This file is copied to ~/spec when you run ''ruby script/generate rspec'' >>> # from the project root directory. >>> ENV["RAILS_ENV"] = "test" >>> require File.expand_path(File.dirname(__FILE__) + >>> "/../config/environment") >>> require ''spec'' >>> require ''spec/rails'' >>> >>> >> This is exactly what the top of my spec_helper.rb file looks like. >> >> I have several specs, and the success of running them is different depending >> on what I do. I''m really just trying to figure out how to take advantage of >> the "rake spec" task. >> I''m on Windows XP. Rails 2.1.0. Rspec and Rspec-Rails 1.1.8 gems. No >> Rspec plugins. Freshly generate rspec stuff using "ruby script/generate >> rspec" >> >> 1) If I run each of my specs separately on the command line using "spec >> spec/whatever.rb" they run fine. >> >> 2) If I run "spec spec/*spec.rb", then all _but one_ run without error (note >> that the one that files can be run successfully using a direct call to it as >> in #1 above). >> >> 3) If I run "rake spec", I get multiple failures, all because the >> config/environments/test.rb file is not loaded (I know this because they >> fail due to the lack of a variable that is set only in test.rb). The spec >> that failed in #2 works in this case. >> >> 4) If I run "rake spec RAILS_ENV=test", all _but one_ run without error >> (same error as in #2). >> >> I draw the following conclustions: >> >> 1) I only get the color output if I use "rake spec" (not sure why) >> >> 2) spec loads test.rb by default, but "rake spec" does not and that''s why I >> have to pass RAILS_ENV=test to the "rake" command. >> > > What is test.rb? >config/environments/test.rb - has test specific stuff set in it.> As for getting different results if you run files individually, with > the spec command or with the rake command, that is *usually* a sign > that there is some state leaking between examples. ''rake spec'' and > ''spec spec'' don''t load the files in the same order, so the fact that > you also get different failures from each of those also supports this > theory in your case. >I agree with this although it is still unclear to me what the state problems are. Thanks, Wes
On Wed, Oct 8, 2008 at 6:42 PM, Wes Gamble <weyus at att.net> wrote:> David Chelimsky wrote: >> >> On Wed, Oct 8, 2008 at 3:41 PM, Wes Gamble <weyus at att.net> wrote: >> >>> >>> Zach, >>> >>> Zach Dennis wrote: >>> >>>> >>>> # This file is copied to ~/spec when you run ''ruby script/generate >>>> rspec'' >>>> # from the project root directory. >>>> ENV["RAILS_ENV"] = "test" >>>> require File.expand_path(File.dirname(__FILE__) + >>>> "/../config/environment") >>>> require ''spec'' >>>> require ''spec/rails'' >>>> >>>> >>> >>> This is exactly what the top of my spec_helper.rb file looks like. >>> >>> I have several specs, and the success of running them is different >>> depending >>> on what I do. I''m really just trying to figure out how to take advantage >>> of >>> the "rake spec" task. >>> I''m on Windows XP. Rails 2.1.0. Rspec and Rspec-Rails 1.1.8 gems. No >>> Rspec plugins. Freshly generate rspec stuff using "ruby script/generate >>> rspec" >>> >>> 1) If I run each of my specs separately on the command line using "spec >>> spec/whatever.rb" they run fine. >>> >>> 2) If I run "spec spec/*spec.rb", then all _but one_ run without error >>> (note >>> that the one that files can be run successfully using a direct call to it >>> as >>> in #1 above). >>> >>> 3) If I run "rake spec", I get multiple failures, all because the >>> config/environments/test.rb file is not loaded (I know this because they >>> fail due to the lack of a variable that is set only in test.rb). The >>> spec >>> that failed in #2 works in this case. >>> >>> 4) If I run "rake spec RAILS_ENV=test", all _but one_ run without error >>> (same error as in #2). >>> >>> I draw the following conclustions: >>> >>> 1) I only get the color output if I use "rake spec" (not sure why) >>> >>> 2) spec loads test.rb by default, but "rake spec" does not and that''s why >>> I >>> have to pass RAILS_ENV=test to the "rake" command. >>> >> >> What is test.rb? >> > > config/environments/test.rb - has test specific stuff set in it. >> >> As for getting different results if you run files individually, with >> the spec command or with the rake command, that is *usually* a sign >> that there is some state leaking between examples. ''rake spec'' and >> ''spec spec'' don''t load the files in the same order, so the fact that >> you also get different failures from each of those also supports this >> theory in your case. >> > > I agree with this although it is still unclear to me what the state problems > are.Based on the other thread you started about the environment - it might be the lack of environments/test.rb being loaded (rather than a state problem). Make sure every _spec.rb file requires spec_helper.rb and see if that solves the problem. Cheers, David> > Thanks, > Wes > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users >
David Chelimsky wrote:> On Wed, Oct 8, 2008 at 3:41 PM, Wes Gamble <weyus at att.net> wrote: > >> Zach, >> >> Zach Dennis wrote: >> >>> # This file is copied to ~/spec when you run ''ruby script/generate rspec'' >>> # from the project root directory. >>> ENV["RAILS_ENV"] = "test" >>> require File.expand_path(File.dirname(__FILE__) + >>> "/../config/environment") >>> require ''spec'' >>> require ''spec/rails'' >>> >>> 1) If I run each of my specs separately on the command line using "spec >>> spec/whatever.rb" they run fine. >>> >>> 2) If I run "spec spec/*spec.rb", then all _but one_ run without error (note >>> that the one that files can be run successfully using a direct call to it as >>> in #1 above). >>> >>> > As for getting different results if you run files individually, with > the spec command or with the rake command, that is *usually* a sign > that there is some state leaking between examples. ''rake spec'' and > ''spec spec'' don''t load the files in the same order, so the fact that > you also get different failures from each of those also supports this > theory in your case. > >I agree with your statement, but I can reproduce a case where I can run all of my specs using the "spec" command using "spec spec/**/*_spec.rb" and one of them will fail. I can either leave in or remove the "--reverse" option in my spec.opts file and get that failure. But...when I run the spec standalone, using "spec spec/<specific_spec_name>.rb" it works fine. So this is a difference in behavior within the spec command and the only difference is that it''s one test (SUCCESS) vs. 22 tests (FAIL). To the best of my knowledge, the other tests don''t even touch the stuff that is under test in the spec that fails, so I can''t figure out where the state would be leaking. (Argh - of course if I reverse the order of the 22 tests, while in both cases the individual spec that I''m talking about always fails, if I reverse the order, one set is a lot more successful than the other. Sigh.) Wes Wes
David Chelimsky wrote:> On Wed, Oct 8, 2008 at 6:42 PM, Wes Gamble <weyus at att.net> wrote: > >> David Chelimsky wrote: >> >>> On Wed, Oct 8, 2008 at 3:41 PM, Wes Gamble <weyus at att.net> wrote: >>> >>> >>>> Zach, >>>> >>>> Zach Dennis wrote: >>>> >>>> >>>>> # This file is copied to ~/spec when you run ''ruby script/generate >>>>> rspec'' >>>>> # from the project root directory. >>>>> ENV["RAILS_ENV"] = "test" >>>>> require File.expand_path(File.dirname(__FILE__) + >>>>> "/../config/environment") >>>>> require ''spec'' >>>>> require ''spec/rails'' >>>>> >>>>> >>>>> >>>> This is exactly what the top of my spec_helper.rb file looks like. >>>> >>>> I have several specs, and the success of running them is different >>>> depending >>>> on what I do. I''m really just trying to figure out how to take advantage >>>> of >>>> the "rake spec" task. >>>> I''m on Windows XP. Rails 2.1.0. Rspec and Rspec-Rails 1.1.8 gems. No >>>> Rspec plugins. Freshly generate rspec stuff using "ruby script/generate >>>> rspec" >>>> >>>> 1) If I run each of my specs separately on the command line using "spec >>>> spec/whatever.rb" they run fine. >>>> >>>> 2) If I run "spec spec/*spec.rb", then all _but one_ run without error >>>> (note >>>> that the one that files can be run successfully using a direct call to it >>>> as >>>> in #1 above). >>>> >>>> 3) If I run "rake spec", I get multiple failures, all because the >>>> config/environments/test.rb file is not loaded (I know this because they >>>> fail due to the lack of a variable that is set only in test.rb). The >>>> spec >>>> that failed in #2 works in this case. >>>> >>>> 4) If I run "rake spec RAILS_ENV=test", all _but one_ run without error >>>> (same error as in #2). >>>> >>>> I draw the following conclustions: >>>> >>>> 1) I only get the color output if I use "rake spec" (not sure why) >>>> >>>> 2) spec loads test.rb by default, but "rake spec" does not and that''s why >>>> I >>>> have to pass RAILS_ENV=test to the "rake" command. >>>> >>>> >>> What is test.rb? >>> >>> >> config/environments/test.rb - has test specific stuff set in it. >> >>> As for getting different results if you run files individually, with >>> the spec command or with the rake command, that is *usually* a sign >>> that there is some state leaking between examples. ''rake spec'' and >>> ''spec spec'' don''t load the files in the same order, so the fact that >>> you also get different failures from each of those also supports this >>> theory in your case. >>> >>> >> I agree with this although it is still unclear to me what the state problems >> are. >> Make sure every _spec.rb file requires spec_helper.rb and see if that >> solves the problem. >> >>Interesting. So I made sure that the first line in every spec file was "require File.join(File.dirname(__FILE__), ''spec_helper.rb'')" [ and I assume that this should be a Cardinal Rule to live by ] and now my 22 tests behave the same way with or without the spec.opts file in play (either in alphabetical order or by mtime and reversed, etc.). 21 tests will pass and the 1 test will fail. Here''s the offending spec. describe "User account fixtures" do it "should exist" do existing=User.all.map(&:login) ["admin", "bor_admin", "dell_admin", "ibm_admin", "ibm_user1", "proc", "van_admin"].each do |login| existing.include?(login).should==true end end end This is pretty straight forward - it''s just testing the presence of a fixture. Now, this fixture is under the test/fixtures directory. Note that I have NOT changed the fixture path in spec_helper.rb from spec/fixtures to text/fixtures, and I have not asked for a fixture to be loaded in any test. However, why would all of my other tests work if no fixtures were ever being loaded? I''m still looking into this. Will report results. Wes
Wes Gamble wrote:> David Chelimsky wrote: >> On Wed, Oct 8, 2008 at 6:42 PM, Wes Gamble <weyus at att.net> wrote: >> >>> David Chelimsky wrote: >>> >>>> On Wed, Oct 8, 2008 at 3:41 PM, Wes Gamble <weyus at att.net> wrote: >>>> >>>> >>>>> Zach, >>>>> >>>>> Zach Dennis wrote: >>>>> >>>>> >>>>>> # This file is copied to ~/spec when you run ''ruby script/generate >>>>>> rspec'' >>>>>> # from the project root directory. >>>>>> ENV["RAILS_ENV"] = "test" >>>>>> require File.expand_path(File.dirname(__FILE__) + >>>>>> "/../config/environment") >>>>>> require ''spec'' >>>>>> require ''spec/rails'' >>>>>> >>>>>> >>>>>> >>>>> This is exactly what the top of my spec_helper.rb file looks like. >>>>> >>>>> I have several specs, and the success of running them is different >>>>> depending >>>>> on what I do. I''m really just trying to figure out how to take >>>>> advantage >>>>> of >>>>> the "rake spec" task. >>>>> I''m on Windows XP. Rails 2.1.0. Rspec and Rspec-Rails 1.1.8 >>>>> gems. No >>>>> Rspec plugins. Freshly generate rspec stuff using "ruby >>>>> script/generate >>>>> rspec" >>>>> >>>>> 1) If I run each of my specs separately on the command line using >>>>> "spec >>>>> spec/whatever.rb" they run fine. >>>>> >>>>> 2) If I run "spec spec/*spec.rb", then all _but one_ run without >>>>> error >>>>> (note >>>>> that the one that files can be run successfully using a direct >>>>> call to it >>>>> as >>>>> in #1 above). >>>>> >>>>> 3) If I run "rake spec", I get multiple failures, all because the >>>>> config/environments/test.rb file is not loaded (I know this >>>>> because they >>>>> fail due to the lack of a variable that is set only in test.rb). The >>>>> spec >>>>> that failed in #2 works in this case. >>>>> >>>>> 4) If I run "rake spec RAILS_ENV=test", all _but one_ run without >>>>> error >>>>> (same error as in #2). >>>>> >>>>> I draw the following conclustions: >>>>> >>>>> 1) I only get the color output if I use "rake spec" (not sure why) >>>>> >>>>> 2) spec loads test.rb by default, but "rake spec" does not and >>>>> that''s why >>>>> I >>>>> have to pass RAILS_ENV=test to the "rake" command. >>>>> >>>>> >>>> What is test.rb? >>>> >>>> >>> config/environments/test.rb - has test specific stuff set in it. >>> >>>> As for getting different results if you run files individually, with >>>> the spec command or with the rake command, that is *usually* a sign >>>> that there is some state leaking between examples. ''rake spec'' and >>>> ''spec spec'' don''t load the files in the same order, so the fact that >>>> you also get different failures from each of those also supports this >>>> theory in your case. >>>> >>>> >>> I agree with this although it is still unclear to me what the state >>> problems >>> are. >>> Make sure every _spec.rb file requires spec_helper.rb and see if that >>> solves the problem. >>> >>> > Interesting. So I made sure that the first line in every spec file was > > "require File.join(File.dirname(__FILE__), ''spec_helper.rb'')" [ and I > assume that this should be a Cardinal Rule to live by ] > > and now my 22 tests behave the same way with or without the spec.opts > file in play (either in alphabetical order or by mtime and reversed, > etc.). 21 tests will pass and the 1 test will fail. > Here''s the offending spec. > > describe "User account fixtures" do > it "should exist" do > existing=User.all.map(&:login) > > ["admin", "bor_admin", "dell_admin", "ibm_admin", "ibm_user1", > "proc", "van_admin"].each do |login| > existing.include?(login).should==true > end > end > end > > This is pretty straight forward - it''s just testing the presence of a > fixture. Now, this fixture is under the test/fixtures directory. > Note that I have NOT changed the fixture path in spec_helper.rb from > spec/fixtures to text/fixtures, and I have not asked for a fixture to > be loaded in any test. However, why would all of my other tests work > if no fixtures were ever being loaded?ANSWER: The other specs are actually using Watir (the IE GUI test framework), and these tests are implicit hitting a server running in development. So, there''s no need to load the data that those test use into the test database using fixtures (we probably need to fix this, etc.). So, my oddball test that was breaking was in fact, the only test hitting the test database that needed any fixture data loaded, and as we see above, it wasn''t getting it. Adding "fixtures :users" in the describe block worked like you would expect. Thanks for all the help, everyone. Wes