So far I''ve been doing my controller testing against "real" (testing) objects; however, I''ve been beginning to wonder if I should "default" to using mocks in most cases -- both for performence and to enforce a looser coupling. In part this is driven by my viewpoint that rail''s "functional" tests are still "unit" tests, so they should be tightly focused on the controller and not on the layers below it. Under this philosophy, it should not be hitting the DB, etc. Now I''m more use to mocks from EasyMock in Java or SimpleTest in PHP which are both of the dynamic mock generation approach, rather than the built-in rails hand-code approach. These other approaches I think make it slightly easier to have widespread use of simple/custom mocks; and I can see how attempting to do the same in Rails (without additional plugins) might be a mistake. Using a tool like Test::Rails from the ZenTest gem would seem to push even stronger towards the mock''d approach as the view and controller tests are seperated and thus more easily recongnizeable as unit tests. How have other people apprached this issue? Do you mock with abandon in the controller tests, or only when absolutely required, or have you found some sweet-spot between the two extremes that you can share? Eric -- Posted via http://www.ruby-forum.com/.
On Jun 14, 2006, at 7:25 AM, Eric D. Nielsen wrote:> How have other people apprached this issue? Do you mock with > abandon in > the controller tests, or only when absolutely required, or have you > found some sweet-spot between the two extremes that you can share?I use mocks to replace methods that cause problems during testing due to their activity being inappropriate for testing, or won''t force the code to journey through the code path I''d like to test (think error handling). A good example of this is credit card processing. You probably don''t want to hit you processing each time you test, and you''d probably like to test error handling if they''re unavailable. With Mocks you can replace the method that actually calls out to them, and have the new one respect an attribute that you can set in the test that causes the mock method to return error codes. As for testing other layers via functional tests, how could that be avoided? Even in unit tests, I don''t have very many methods that don''t depend on other methods I''ve written, so it''s very rare when tests only run through a single method. -- -- Tom Mornini
I found out that this paper provides food rules of thumb (at least for me!) to decide to go mock or not: http://www.pragmaticprogrammer.com/articles/may_02_mock.pdf Use mock when... The real object has nondeterministic behavior. The real object is difficult to set up. The real object has behavior that is hard to trigger (for example, a network error). The real object is slow. The real object has (or is) a user interface. The test needs to ask the real object about how it was used (for example, a test might need to check to see that a callback function was actually called). The real object does not yet exist. Thibaut -------------- next part -------------- An HTML attachment was scrubbed... URL: http://wrath.rubyonrails.org/pipermail/rails/attachments/20060614/caabff6d/attachment.html
> provides food rules of thumb (at least for me!) to decide to go mock ornot: sub(''food'',''good'') !!! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://wrath.rubyonrails.org/pipermail/rails/attachments/20060614/c7040c3a/attachment.html
Tom Mornini wrote:> On Jun 14, 2006, at 7:25 AM, Eric D. Nielsen wrote: > >> How have other people apprached this issue? Do you mock with >> abandon in >> the controller tests, or only when absolutely required, or have you >> found some sweet-spot between the two extremes that you can share? > > I use mocks to replace methods that cause problems during testing due to > their activity being inappropriate for testing, or won''t force the code > to journey through the code path I''d like to test (think error > handling). > > A good example of this is credit card processing. You probably don''t > want to hit you processing each time you test, and you''d probably like > to test error handling if they''re unavailable. With Mocks you can > replace the method that actually calls out to them, and have the new > one respect an attribute that you can set in the test that causes the > mock method to return error codes.I agree that both your examples above (forcing hard to force error conditions or tests depending on third party network services) are places where mocks are absolutely required.> > As for testing other layers via functional tests, how could that be > avoided? Even in unit tests, I don''t have very many methods that don''t > depend on other methods I''ve written, so it''s very rare when tests > only run through a single method.Well in some sense it comes back to the "Humble Dialog" approach used for testing user-interfaces in either web or desktop applications. Assuming you''ve pushed functionality to "lowest" level (Model < Controller < View) the tests required at the higher levels become successively more trivial. Tightly focused unit tests will often only exercise a single method, but even if they don''t they shouldn''t be exercising code "too far" removed from the object under test. I often use the "Law of Demeter" as a guideline for determining the appropriate "reach" of testing. As to how this can be acheived, lets take the example of a simple "login" action in some controller: If following TDD, we''d normally drive the development with two tests: 1. If the login fails, did we do the appropriate thing 2. If the login succeeds, did we do the appropriate thing. Lets assume that the User model will handle logins. As a client of the User we don''t care how they authenticate, only that they do so. As a result a mock of User with configurable return values (Mock as Actor) can easily sub-in for the actual User without a need for hitting the database. So we could use a Mock like (hypthetical code, not tested for actual run-ability) class User #note does not extend ActiveRecord::Base def self.set_expected_return_for_login(value) @@return_val = value end def login(username, password) @@return_val end end Now we only use the "published" api relevant to us at our controller test. (Using class variables to avoid issues with figuring out a good/simple way to handle the mock injection in this case.) With test code like def successful_login User.set_expected_return_for_login(true) post :action, {:username=>''foo'',:password=>''bar'') assert_redirected_to :where_ever assert .... end As a result you force the path you want to test; you don''t rely on the details of how the lower layer generates the response. Of course, most mock frameworks have improved support to automate the process of setting return values/expectations on methods and do some checking to make sure that the method being knocked out exists in the production code and that the parameter lists match, etc. Thus the mock can detect if you api changes in many cases. Of course you also have your integration/acceptence tests that also help to catch places where you mocks masked an api change (say if you flipped the order of two arguements in a function call) Taken to an extreme this would lead to the controller tests being de-coupled from the database and never needing to use the DB, fixtures, etc... -- Posted via http://www.ruby-forum.com/.
I have a test that grabs signups in the last week, aggregated by day. If I go against a database, I have to fix the database in time and fool the system into thinking the current date is different from today. However, to test the aggregation, I need to have a fair amount of data. How is this scenario best handled? Mocks? Thanks, Steve -- View this message in context: http://www.nabble.com/Controller-testing-and-mocks...-t1786480.html#a4869333 Sent from the RubyOnRails Users forum at Nabble.com.
Steve Ross wrote:> I have a test that grabs signups in the last week, aggregated by day. If > I go > against a database, I have to fix the database in time and fool the > system > into thinking the current date is different from today. However, to test > the > aggregation, I need to have a fair amount of data. > > How is this scenario best handled? Mocks? > > Thanks, > > SteveIt depends :). If its a controller based action, I''ld probably use mocks (but that''s what I would have done in different languages, I started this thread to learn more about how rails developers tend to use them...) If you''re talking about testing the model level code that generates the list for the controller/view, then I''d use dynamic fixtures. -- Posted via http://www.ruby-forum.com/.
On Wed, Jun 14, 2006 at 10:29:26AM -0700, s.ross wrote:> > I have a test that grabs signups in the last week, aggregated by day. If I go > against a database, I have to fix the database in time and fool the system > into thinking the current date is different from today. However, to test the > aggregation, I need to have a fair amount of data. > > How is this scenario best handled? Mocks?Dynamic fixtures, as mentioned. An example: my_drifting_object: id: 3 name: Fred Nerk # 36 hours ago signup_date: <%= (Time.now() - 60 * 60 * 36).strftime(''%Y-%m-%d %H:%M:%S'') %> The only issue I''ve had with this is that, if you want to test boundary conditions, there can be a few seconds between when this fixture data is inserted and when your test runs, so if you set your signup_date to the last second you want to catch, it might end up dropping off the list if your test is a bit slow in running. - Matt
On Wed, Jun 14, 2006 at 04:25:24PM +0200, Eric D. Nielsen wrote:> In part this is driven by my viewpoint that rail''s "functional" tests > are still "unit" tests, so they should be tightly focused on the > controller and not on the layers below it. Under this philosophy, it > should not be hitting the DB, etc.So you''re going to mock out your model objects for your functional tests? In the words of a famous computer, "DANGER! Danger, Will Robinson!". This policy is almost certainly going to cause you Pain and Suffering, as your models will change, but since your controllers are tested using mocks which don''t change, your controllers will still test OK but will fail when actually used. A good rule of thumb is: Mock as little as possible. Never, ever, mock anything you don''t absolutely have to, because you have to then maintain two codebases in parallel, and if they get out of sync you''re stuffed.> Now I''m more use to mocks from EasyMock in Java or SimpleTest in PHP > which are both of the dynamic mock generation approach, rather than the > built-in rails hand-code approach.I did the hand-code thing for a while, but I yearned for SimpleTest''s call-chain verification, so now I''m using Test::Unit::MockObject. It is a fantastic mock object library -- dynamic as all hell, like Ruby itself, and not nearly as verbose as the Python mock library I used a while ago (whose name I have mercifully forgotten). I like T::U::MockObject so much I added a couple of major improvements to it -- I''ve sent them upstream, but a new release hasn''t come out yet. Let me know if you''d like my modified version in the interim.> How have other people apprached this issue? Do you mock with abandon in > the controller tests, or only when absolutely required, or have you > found some sweet-spot between the two extremes that you can share?Mocks are essentially an acknowledgement that you''ve failed in your quest for perfect testing. I''m strong enough to acknowledge my failures, but I still don''t like failing more often than I have to. - Matt
Matthew Palmer wrote:> On Wed, Jun 14, 2006 at 04:25:24PM +0200, Eric D. Nielsen wrote: >> In part this is driven by my viewpoint that rail''s "functional" tests >> are still "unit" tests, so they should be tightly focused on the >> controller and not on the layers below it. Under this philosophy, it >> should not be hitting the DB, etc. > > So you''re going to mock out your model objects for your functional > tests? > In the words of a famous computer, "DANGER! Danger, Will Robinson!". > This > policy is almost certainly going to cause you Pain and Suffering, as > your > models will change, but since your controllers are tested using mocks > which > don''t change, your controllers will still test OK but will fail when > actually used.Well it doesn''t have to be as dangerous as you point out (but it can be). In most of my previous non-Rails based projects, I''ve used a more mapping based ORM layer. Thus I can use "real" model objects, while using a mock for the ORM to avoid the database access overhead. I haven''t figured out how/if to apply this to ActiveRecord style Object/Relational systems, as the database access is intrinsic to the class. As mentioned in the my OP, I feel that the Law of Demeter should cover these situations -- if using a test double to replace an object at two removes from the class under test lets you get into a situation where you have passing unit tests, but failing integration/acceptence tests, that just means you are missing unit tests on the class mediating in the delegation. I''ll have to check out T:U:MockObject, thanks for the hint. Eric -- Posted via http://www.ruby-forum.com/.