I think I have a fundamental misunderstanding of what ''rake test:functionals'' does for me. I assumed it did the exact same thing for me that ''rake test'' did, except that it only ran the functional tests. It seems like this was a bad assumption because in the project I''m working on, when I run ''rake test'', I get no failures or errors, but when I run ''rake test:functionals'', I get failures. Is there something different about what these things do in terms of fixture loading or pre-test setup that I''m not seeing? TIA... Bryan --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
I don''t have an answer for you, Bryan, but I too am curious about rake-related testing differences. My problem is that if I run one of my tests as follows ruby test/unit/box_test.rb it all passes, but if I run "rake test" or just simply "rake" the same test fails. Thanks in advance, whoever can shed light on this problem! Ben On 9/4/07, Bryan Noll <bwnoll-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> > I think I have a fundamental misunderstanding of what ''rake > test:functionals'' does for me. > > I assumed it did the exact same thing for me that ''rake test'' did, > except that it only ran the functional tests. It seems like this was a > bad assumption because in the project I''m working on, when I run ''rake > test'', I get no failures or errors, but when I run ''rake > test:functionals'', I get failures. > > Is there something different about what these things do in terms of > fixture loading or pre-test setup that I''m not seeing? > > TIA... > > Bryan > > > >--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Here''s what was wrong with mine. The functional test that was failing when I ran ''rake test:functionals'' has a long list of fixtures it is using. fixtures :f1, :f2, :f3 I looked closer at what the actual test method was doing, and realized that it was dealing with a type of model object whose fixture did not appear in the list of those I was including. So, I changed that line to this: fixtures :f1, :f2, :f3, :f4 ... and all is well now. So... that fixed my problem, but now I''m a little scared because something else I assumed was happening a certain way seems not to be. Since this particular functional test was passing when I simply ran ''rake test'', I can deduce that one of my functional tests was including the '':f4'' fixture, and that data was still laying around in the database, enabling that functional test to eventually pass. This scares me because I assumed the fixtures would be loaded and wiped away for each and every test so that the state of the database was known. Is this a bad assumption? Ben Nevile wrote:> I don''t have an answer for you, Bryan, but I too am curious about > rake-related testing differences. My problem is that if I run one of > my tests as follows > > ruby test/unit/box_test.rb > > it all passes, but if I run "rake test" or just simply "rake" the same > test fails. > > Thanks in advance, whoever can shed light on this problem! > Ben > > > > > On 9/4/07, Bryan Noll <bwnoll-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote: > >> I think I have a fundamental misunderstanding of what ''rake >> test:functionals'' does for me. >> >> I assumed it did the exact same thing for me that ''rake test'' did, >> except that it only ran the functional tests. It seems like this was a >> bad assumption because in the project I''m working on, when I run ''rake >> test'', I get no failures or errors, but when I run ''rake >> test:functionals'', I get failures. >> >> Is there something different about what these things do in terms of >> fixture loading or pre-test setup that I''m not seeing? >> >> TIA... >> >> Bryan >> >> > > > > >--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Correcting something inline... Bryan Noll wrote:> Here''s what was wrong with mine. > > The functional test that was failing when I ran ''rake > test:functionals'' has a long list of fixtures it is using. > > fixtures :f1, :f2, :f3 > > I looked closer at what the actual test method was doing, and realized > that it was dealing with a type of model object whose fixture did not > appear in the list of those I was including. So, I changed that line > to this: > > fixtures :f1, :f2, :f3, :f4 > > ... and all is well now. > > So... that fixed my problem, but now I''m a little scared because > something else I assumed was happening a certain way seems not to be. > > Since this particular functional test was passing when I simply ran > ''rake test'', I can deduce that one of my functional testsscratch ''functional tests'', should say ''unit tests''> was including the '':f4'' fixture, and that data was still laying around > in the database, enabling that functional test to eventually pass. > > This scares me because I assumed the fixtures would be loaded and > wiped away for each and every test so that the state of the database > was known. Is this a bad assumption? > > Ben Nevile wrote: >> I don''t have an answer for you, Bryan, but I too am curious about >> rake-related testing differences. My problem is that if I run one of >> my tests as follows >> >> ruby test/unit/box_test.rb >> >> it all passes, but if I run "rake test" or just simply "rake" the same >> test fails. >> >> Thanks in advance, whoever can shed light on this problem! >> Ben >> >> >> >> >> On 9/4/07, Bryan Noll <bwnoll-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote: >> >>> I think I have a fundamental misunderstanding of what ''rake >>> test:functionals'' does for me. >>> >>> I assumed it did the exact same thing for me that ''rake test'' did, >>> except that it only ran the functional tests. It seems like this was a >>> bad assumption because in the project I''m working on, when I run ''rake >>> test'', I get no failures or errors, but when I run ''rake >>> test:functionals'', I get failures. >>> >>> Is there something different about what these things do in terms of >>> fixture loading or pre-test setup that I''m not seeing? >>> >>> TIA... >>> >>> Bryan >>> >>> >> >> >> >> >> >--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Well for one thing, if you run ''rake test" all your unit tests run before your functional tests. When you run ''rake test:functionals'' your unit tests don''t run. So I would suspect that your unit tests are loading something (fixtures or whatever) that are not getting loaded when you run functional tests alone. That''s the kind of thing I would look for first. Even try running your tests individually and see if they pass. If they don''t, you''ll likely find that your aren''t loading something you need in the individual test that is failing. On Sep 4, 6:00 pm, "Ben Nevile" <ben.nev...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> I don''t have an answer for you, Bryan, but I too am curious about > rake-related testing differences. My problem is that if I run one of > my tests as follows > > ruby test/unit/box_test.rb > > it all passes, but if I run "rake test" or just simply "rake" the same > test fails. > > Thanks in advance, whoever can shed light on this problem! > Ben > > On 9/4/07, Bryan Noll <bwn...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote: > > > > > I think I have a fundamental misunderstanding of what ''rake > > test:functionals'' does for me. > > > I assumed it did the exact same thing for me that ''rake test'' did, > > except that it only ran the functional tests. It seems like this was a > > bad assumption because in the project I''m working on, when I run ''rake > > test'', I get no failures or errors, but when I run ''rake > > test:functionals'', I get failures. > > > Is there something different about what these things do in terms of > > fixture loading or pre-test setup that I''m not seeing? > > > TIA... > > > Bryan--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Thanks Robert. This is exactly what it turned out to be (see my response in the thread about 3 minutes before yours). Do you have any insight to my concern that I''m not getting a clean slate/known state after unit tests have run? Robert Walker wrote:> Well for one thing, if you run ''rake test" all your unit tests run > before your functional tests. When you run ''rake test:functionals'' > your unit tests don''t run. > > So I would suspect that your unit tests are loading something > (fixtures or whatever) that are not getting loaded when you run > functional tests alone. > > That''s the kind of thing I would look for first. Even try running your > tests individually and see if they pass. If they don''t, you''ll likely > find that your aren''t loading something you need in the individual > test that is failing. > > On Sep 4, 6:00 pm, "Ben Nevile" <ben.nev...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote: > >> I don''t have an answer for you, Bryan, but I too am curious about >> rake-related testing differences. My problem is that if I run one of >> my tests as follows >> >> ruby test/unit/box_test.rb >> >> it all passes, but if I run "rake test" or just simply "rake" the same >> test fails. >> >> Thanks in advance, whoever can shed light on this problem! >> Ben >> >> On 9/4/07, Bryan Noll <bwn...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote: >> >> >> >> >>> I think I have a fundamental misunderstanding of what ''rake >>> test:functionals'' does for me. >>> >>> I assumed it did the exact same thing for me that ''rake test'' did, >>> except that it only ran the functional tests. It seems like this was a >>> bad assumption because in the project I''m working on, when I run ''rake >>> test'', I get no failures or errors, but when I run ''rake >>> test:functionals'', I get failures. >>> >>> Is there something different about what these things do in terms of >>> fixture loading or pre-test setup that I''m not seeing? >>> >>> TIA... >>> >>> Bryan >>> > > > > > >--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
It appears to me from my testing that the test database tables are emptied at the beginning of tests but are not emptied after the test completes. Running functional tests that do not load the fixture in question does not empty the databases table in question. Therefore, a functional test reading data from a table where the fixture is not loaded by that specific functional test will use the data that may have been left behind from a previous test run. Hope that''s not too confusing. It basically boils down to ensuring that you are loading fixtures for all tables accessed by that specific test. On Sep 4, 6:44 pm, Bryan Noll <bwn...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> Thanks Robert. This is exactly what it turned out to be (see my > response in the thread about 3 minutes before yours). Do you have any > insight to my concern that I''m not getting a clean slate/known state > after unit tests have run? > > Robert Walker wrote: > > Well for one thing, if you run ''rake test" all your unit tests run > > before your functional tests. When you run ''rake test:functionals'' > > your unit tests don''t run. > > > So I would suspect that your unit tests are loading something > > (fixtures or whatever) that are not getting loaded when you run > > functional tests alone. > > > That''s the kind of thing I would look for first. Even try running your > > tests individually and see if they pass. If they don''t, you''ll likely > > find that your aren''t loading something you need in the individual > > test that is failing. > > > On Sep 4, 6:00 pm, "Ben Nevile" <ben.nev...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote: > > >> I don''t have an answer for you, Bryan, but I too am curious about > >> rake-related testing differences. My problem is that if I run one of > >> my tests as follows > > >> ruby test/unit/box_test.rb > > >> it all passes, but if I run "rake test" or just simply "rake" the same > >> test fails. > > >> Thanks in advance, whoever can shed light on this problem! > >> Ben > > >> On 9/4/07, Bryan Noll <bwn...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote: > > >>> I think I have a fundamental misunderstanding of what ''rake > >>> test:functionals'' does for me. > > >>> I assumed it did the exact same thing for me that ''rake test'' did, > >>> except that it only ran the functional tests. It seems like this was a > >>> bad assumption because in the project I''m working on, when I run ''rake > >>> test'', I get no failures or errors, but when I run ''rake > >>> test:functionals'', I get failures. > > >>> Is there something different about what these things do in terms of > >>> fixture loading or pre-test setup that I''m not seeing? > > >>> TIA... > > >>> Bryan--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Ugg. And I thought that the builtin test framework was bad enough already. What this means is that your tests need to do complete teardowns-- including the test database. I''ve had a number of cases where I mocked behavior for a test and then had the mocking interfere with the testing of the real function. I tend to be pretty aggressive about creating modules to test inside now. This sounds like another time where best practice is going to have to be learned the hard way. On Sep 4, 6:18 pm, Robert Walker <rwalker...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> It appears to me from my testing that the test database tables are > emptied at the beginning of tests but are not emptied after the test > completes. Running functional tests that do not load the fixture in > question does not empty the databases table in question. Therefore, a > functional test reading data from a table where the fixture is not > loaded by that specific functional test will use the data that may > have been left behind from a previous test run. > > Hope that''s not too confusing. It basically boils down to ensuring > that you are loading fixtures for all tables accessed by that specific > test. > > On Sep 4, 6:44 pm, Bryan Noll <bwn...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote: > > > Thanks Robert. This is exactly what it turned out to be (see my > > response in the thread about 3 minutes before yours). Do you have any > > insight to my concern that I''m not getting a clean slate/known state > > after unit tests have run? > > > Robert Walker wrote: > > > Well for one thing, if you run ''rake test" all your unit tests run > > > before your functional tests. When you run ''rake test:functionals'' > > > your unit tests don''t run. > > > > So I would suspect that your unit tests are loading something > > > (fixtures or whatever) that are not getting loaded when you run > > > functional tests alone. > > > > That''s the kind of thing I would look for first. Even try running your > > > tests individually and see if they pass. If they don''t, you''ll likely > > > find that your aren''t loading something you need in the individual > > > test that is failing. > > > > On Sep 4, 6:00 pm, "Ben Nevile" <ben.nev...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote: > > > >> I don''t have an answer for you, Bryan, but I too am curious about > > >> rake-related testing differences. My problem is that if I run one of > > >> my tests as follows > > > >> ruby test/unit/box_test.rb > > > >> it all passes, but if I run "rake test" or just simply "rake" the same > > >> test fails. > > > >> Thanks in advance, whoever can shed light on this problem! > > >> Ben > > > >> On 9/4/07, Bryan Noll <bwn...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote: > > > >>> I think I have a fundamental misunderstanding of what ''rake > > >>> test:functionals'' does for me. > > > >>> I assumed it did the exact same thing for me that ''rake test'' did, > > >>> except that it only ran the functional tests. It seems like this was a > > >>> bad assumption because in the project I''m working on, when I run ''rake > > >>> test'', I get no failures or errors, but when I run ''rake > > >>> test:functionals'', I get failures. > > > >>> Is there something different about what these things do in terms of > > >>> fixture loading or pre-test setup that I''m not seeing? > > > >>> TIA... > > > >>> Bryan--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
On 9/4/07, Bryan Noll <bwnoll-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> > I think I have a fundamental misunderstanding of what ''rake > test:functionals'' does for me. > > I assumed it did the exact same thing for me that ''rake test'' did, > except that it only ran the functional tests. It seems like this was a > bad assumption because in the project I''m working on, when I run ''rake > test'', I get no failures or errors, but when I run ''rake > test:functionals'', I get failures. > > Is there something different about what these things do in terms of > fixture loading or pre-test setup that I''m not seeing?You''ve discovered that it''s fixture-related. You have a test case that is using a fixture that you aren''t explicitly loading, but that a different test case *is* loading. When you run the test cases in different orders, you can get strange errors. The solution is to specify *all* your fixtures in each test case. I have the following method in test_helper.rb: def self.all_fixtures Dir.glob(File.join(File.dirname(__FILE__), ''fixtures'', ''*.{yml,csv}'')).each do |fixture_file| fixtures File.basename(fixture_file, ''.*'') end end Then in each test case, instead of listing specific fixture names, I just use: all_fixtures --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
IMHO, this will land you right back in the soup. Different tests will have different needs for fixtures. And edge testing conditions tend (very much) to be incompatible with each other. Certainly, you might have a base set of fixtures, or even a collection which might be addressed as an array, but the topology is likely to be complex overall. YMMV and all, but you are far less likely to run into trouble if you load precisely the fixtures that you intend for each test or set of tests--and clean up when you are done. On Sep 5, 8:38 am, "Bob Showalter" <showa...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> On 9/4/07, Bryan Noll <bwn...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote: > > > > > I think I have a fundamental misunderstanding of what ''rake > > test:functionals'' does for me. > > > I assumed it did the exact same thing for me that ''rake test'' did, > > except that it only ran the functional tests. It seems like this was a > > bad assumption because in the project I''m working on, when I run ''rake > > test'', I get no failures or errors, but when I run ''rake > > test:functionals'', I get failures. > > > Is there something different about what these things do in terms of > > fixture loading or pre-test setup that I''m not seeing? > > You''ve discovered that it''s fixture-related. You have a test case that > is using a fixture that you aren''t explicitly loading, but that a > different test case *is* loading. When you run the test cases in > different orders, you can get strange errors. > > The solution is to specify *all* your fixtures in each test case. I > have the following method in test_helper.rb: > > def self.all_fixtures > Dir.glob(File.join(File.dirname(__FILE__), ''fixtures'', ''*.{yml,csv}'')).each > do |fixture_file| > fixtures File.basename(fixture_file, ''.*'') > end > end > > Then in each test case, instead of listing specific fixture names, I just use: > > all_fixtures--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---