I have decided that for now I am going to use the authlogic gem to provide authentication for the project I am creating. The question arises, since authentication is a user feature request and since it is to be satisfied through the use of a gem (not a plugin) then should one test for the availability of the gem or not? Secondly, given that a gem is providing the authentication logic (and is not my code which therefore should not be tested by me) then what should one test for with respect to the specific implementation? The existence of particular attributes? The existence of a users model? Does one write step definitions (assuming rspec) like: When /users are authenticated/ do #TODO: This does not work even when the model class exists. User.should exist end When /authentication by authlogic/ do #TODO: Find out how to test for a loaded gem within an application end -- Posted via http://www.ruby-forum.com/.
On 9 Dec 2008, at 19:06, James Byrne wrote:> I have decided that for now I am going to use the authlogic gem to > provide authentication for the project I am creating. The question > arises, since authentication is a user feature request and since it is > to be satisfied through the use of a gem (not a plugin) then should > one > test for the availability of the gem or not? > > Secondly, given that a gem is providing the authentication logic > (and is > not my code which therefore should not be tested by me) then what > should > one test for with respect to the specific implementation? The > existence > of particular attributes? The existence of a users model? > > Does one write step definitions (assuming rspec) like: > > When /users are authenticated/ do > #TODO: This does not work even when the model class exists. > User.should exist > end > > When /authentication by authlogic/ do > #TODO: Find out how to test for a loaded gem within an application > endIf you''re working at the level of Cucumber features, I would stay well away from implementation details like the particular technology you''re using for authentication. Remember this is called Behaviour Driven Development[1] for a reason. You''re focussing on specifying the desired *behaviour* of the system. How you chose to satisfy the user''s need for that behaviour is an entirely separate matter. A large benefit of having Acceptance Tests, IMO, is that you could radically change your implementation under the hood, such as switching to a different authentication mechanism, and the features would still be valid. You might have to code up some of your steps a little differently, but as long as the users want the same thing from the system, the features should not have to change in the slightest. [1]http://dannorth.net/introducing-bdd Matt Wynne http://blog.mattwynne.net http://www.songkick.com
Matt Wynne wrote:> On 9 Dec 2008, at 19:06, James Byrne wrote: > >> should >> >> When /authentication by authlogic/ do >> #TODO: Find out how to test for a loaded gem within an application >> end > > If you''re working at the level of Cucumber features, I would stay well > away from implementation details like the particular technology you''re > using for authentication. >The features for this simply say something along the lines of: Scenario: Application has known users Given that the application has users When the user signs on Then the user is authenticated> Remember this is called Behaviour Driven Development[1] for a reason. > You''re focussing on specifying the desired *behaviour* of the system. > How you chose to satisfy the user''s need for that behaviour is an > entirely separate matter.Nonetheless, in the step definitions I must test the behaviour in a fashion which first drives and then confirms the implementation details. If the chosen implementation depends upon a third party plugin or gem then surely one should test for its provision? What is the alternative? Write tests to test that gems behaviour? Does that not violate TDD/BDD standard of only testing your own code?> > A large benefit of having Acceptance Tests, IMO, is that you could > radically change your implementation under the hood, such as switching > to a different authentication mechanism, and the features would still > be valid. You might have to code up some of your steps a little > differently, but as long as the users want the same thing from the > system, the features should not have to change in the slightest. >If the implementation changed then the step definitions must change to suit, must they not? One is, in the end, only testing what one actually does or wishes done. I do not see how in this case any implementation change would necessitate revising the features given above. If we were implementing our own authentication system then clearly more features or scenarios might be specified but the existing ones need not change.> [1]http://dannorth.net/introducing-bddRead, re-read, and re-re-read. Nonetheless, it is possible to become dogmatic over such things. There is a considerable, and growing, body of opinion that so-called acceptance testing is suitable for many, possibly, all levels of application implementation. That the key to understanding when to use AT really lies in a flexible interpretation of the role of user. This is, as has been pointed out to me many times, an evolving art. The imaginary end user of any non-trivial application is actually an assemblage of disparate roles, many of which consciously do not overlap. Some of those users are the system administrators which tend to the day to day background maintenance tasks, some of which are automated and some of which are not. Some of the application users are, in truth, the implementors themselves. Not all features are of concern to all users/roles. Why should some users be denied the use of features and BDD as a tool for their own concerns simply because they are writing the code? It seems to me an artificial, and logically insupportable, distinction to make. While I am still a novice with much of this specific technology (TDD/BDD/Ruby/Rails) I have had a lot of experience, much of it bad, with overly rigid, and often misinformed, interpretation and employment of design methodologies. The quest for the silver bullet has led many teams down methodological dead ends, whereas a more flexible use of the same tool might have produced a beneficial outcome. What I have come up with is this, for now: When /has known users/ do # If we have known users then they must be stored somewhere my_user = User.new end When /user logs in/ do # If a user signs on then they do it here visits login_path end When /user signs on/ do When "user logs in" end When /user "(.*)" is authenticated/ do |u| # We use Authlogic for authentication so just test that gem is loaded assert defined?("Authlogic") end -- Posted via http://www.ruby-forum.com/.
James Byrne wrote:
Of course, the last example should have been:
When /user is authenticated/ do
# We use Authlogic for authentication so just test that gem is loaded
assert defined?("Authlogic")
end
--
Posted via http://www.ruby-forum.com/.
On 10 Dec 2008, at 16:04, James Byrne wrote:> Nonetheless, in the step definitions I must test the behaviour in a > fashion which first drives and then confirms the implementation > details.This is not the path to BDD, it''s the path to state-based testing. Matt has already explained this well: "How you chose to satisfy the user''s need for that behaviour is an entirely separate matter. A large benefit of having Acceptance Tests, IMO, is that you could radically change your implementation under the hood, such as switching to a different authentication mechanism, and the features would still be valid." I''m guessing the customer for this project didn''t ask for AuthLogic, they asked for part of the app to be protected. You should only write acceptance tests for things these users care about.> If the chosen implementation depends upon a third party plugin or gem > then surely one should test for its provision? What is the > alternative? > Write tests to test that gems behaviour? Does that not violate TDD/ > BDD > standard of only testing your own code?Don''t confuse library code with code you write to use this library. The AuthLogic config in your app is code too, and it adds behaviours to your app. The fact there are 10, 100 or 10,000 lines of code behind it is irrelevant. There''s two attitudes you can take in spec files: write specs that your code calls some library, or write specs that describe the behaviour that library provides. For example (the following is rushed, contrived code but illustrates the idea): class Sorter def initialize(array) @array = array end def sort array.sort end end now when you spec @array = [5,9,6,1,4] @sorter = Sorter.new(@array) are you going to write @array.should_receive(:sort) or @sorter.sort.should == [1,4,5,6,9] ? I believe that in this case, the second option is more valuble: you can change the implementation to sort other collections. In the first case, you could refactor the code without breaking the existing spec. This is important: *The ability to refactor code without breaking the specs is one of the major sources of value in the specs*. This lets you reduce technical debt with no risk to your app. If you rely on the implementation in your spec, you have to change the spec, and therefore you have no guarantee that the behaviour your users *asked for* still works. The definition of refactoring is changing the implementation of code without changing its behaviour (and by extension its specs). Without this constant your code will be fragile. As a extreme, albeit small, example- a while back I took the code for a presentation I gave[1] and changed the app framework from Ramaze to Merb (it only took about 10 mins). Because the specs all ran across the public HTML interface, there was no visible difference. If the feature steps relied on the implementation this wouldn''t have been possible. I use Celerity[2] instead of Webrat and write all steps across an app''s public interface for this reason.> The imaginary end user of any non-trivial application is actually an > assemblage of disparate roles, many of which consciously do not > overlap. > Some of those users are the system administrators which tend to the > day > to day background maintenance tasks, some of which are automated and > some of which are not. Some of the application users are, in truth, > the > implementors themselves.This is fine, but you should still write acceptance tests from the perspective of the user. If the user is a shell script, drive your app on the command line. If the user is a human, drive it over the visible interface. But do you really have any users that access your app code directly? Hope this gives another perspective on the subject. Ashley [1] http://aviewfromafar.net/2008/10/2/geekup-sheffield-vi-from-specification-to-success [2] http://celerity.rubyforge.org/ -- http://www.patchspace.co.uk/ http://aviewfromafar.net/
Ashley Moran wrote:> > Don''t confuse library code with code you write to use this library. > The AuthLogic config in your app is code too, and it adds behaviours > to your app. The fact there are 10, 100 or 10,000 lines of code > behind it is irrelevant.So, if I understand you correctly, then I should write features somewhat like this: Scenario: A known user signs in successfully Given a known user "myuser" with a password "myuserpassword" When user "myuser" signs on with a password "myuserpassword" Then user "myuser" is authenticated And the user sees a sign in success message Scenario: A known user does not sign in successfully Given a known user "myuser" with a password "myuserpassword" When user "myuser" signs in with a password "anotherpassword" Then user "myuser" is not authenticated And the user sees a sign in failure message And the user sees the sign in page And I should simply not worry about what is provided by authlogic, other than the requisite behaviour as exhibited in this application.> > Hope this gives another perspective on the subject. >I am still grappling with the entire concept. A "perspective" graces what I as yet possess with too much dignity. Thanks. -- Posted via http://www.ruby-forum.com/.
On 10 Dec 2008, at 16:26, James Byrne wrote:> James Byrne wrote: > Of course, the last example should have been: > > When /user is authenticated/ do > # We use Authlogic for authentication so just test that gem is loaded > assert defined?("Authlogic") > endTo verify that the user has been authenticated, why not try something like Then /I should see that I am logged in/ do user = User.first response.should include_text "welcome, #{user.username}" end This is much less coupled to your implementation, though it does assume that the ''I'' referred to in the step is the only User in your database, or at least happens to be the first one. I''ll leave it as a exercise to the reader to work around that one. Matt Wynne http://blog.mattwynne.net http://www.songkick.com
I put a starter application with a bunch of features to test authorisation code implemented by Restful Authentication on Github a few days ago. In theory you could apply these features directly to your Authlogic implementation. http://github.com/diabolo/fbrp/tree/master All best Andrew 2008/12/10 James Byrne <lists at ruby-forum.com>> Ashley Moran wrote: > > > > > Don''t confuse library code with code you write to use this library. > > The AuthLogic config in your app is code too, and it adds behaviours > > to your app. The fact there are 10, 100 or 10,000 lines of code > > behind it is irrelevant. > > So, if I understand you correctly, then I should write features somewhat > like this: > > Scenario: A known user signs in successfully > Given a known user "myuser" with a password "myuserpassword" > When user "myuser" signs on with a password "myuserpassword" > Then user "myuser" is authenticated > And the user sees a sign in success message > > Scenario: A known user does not sign in successfully > Given a known user "myuser" with a password "myuserpassword" > When user "myuser" signs in with a password "anotherpassword" > Then user "myuser" is not authenticated > And the user sees a sign in failure message > And the user sees the sign in page > > And I should simply not worry about what is provided by authlogic, other > than the requisite behaviour as exhibited in this application. > > > > > > Hope this gives another perspective on the subject. > > > > I am still grappling with the entire concept. A "perspective" graces > what I as yet possess with too much dignity. > > Thanks. > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://rubyforge.org/pipermail/rspec-users/attachments/20081211/425a2cc2/attachment.html>
On 10 Dec 2008, at 17:40, James Byrne wrote:> So, if I understand you correctly, then I should write features > somewhat > like this: > > Scenario: A known user signs in successfully > Given a known user "myuser" with a password "myuserpassword" > When user "myuser" signs on with a password "myuserpassword" > Then user "myuser" is authenticated > And the user sees a sign in success message > > Scenario: A known user does not sign in successfully > Given a known user "myuser" with a password "myuserpassword" > When user "myuser" signs in with a password "anotherpassword" > Then user "myuser" is not authenticated > And the user sees a sign in failure message > And the user sees the sign in page > > And I should simply not worry about what is provided by authlogic, > other > than the requisite behaviour as exhibited in this application.That''s how I (and many others here) tackle it. But there''s no need for the steps Then user "myuser" is authenticated and Then user "myuser" is not authenticated because these don''t describe behaviour, they describe state. The "sign in message" steps are what you really want. And you can implement these as Matt described. Since you''ve written steps that create the user (preferably through the app itself!), you could extend the step like this Given a known user "Mr Known User" with credentials "myuser" : "myuserpassword" Then you already have the name to verify if you want to display a personalised message. (On the other hand, putting too much data in the steps gets cumbersome; I tend to write them more like this: Scenario: A known user signs in successfully Given I am a registered user When I sign on with the correct credentials Then I should see a sign in success message Even if internally I call out to steps more like the ones you wrote as examples. Hope that doesn''t muddy the waters though. Ashley -- http://www.patchspace.co.uk/ http://aviewfromafar.net/
Ashley Moran wrote:> > Then you already have the name to verify if you want to display a > personalised message. (On the other hand, putting too much data in > the steps gets cumbersome; I tend to write them more like this: > > Scenario: A known user signs in successfully > Given I am a registered user > When I sign on with the correct credentials > Then I should see a sign in success message > > Even if internally I call out to steps more like the ones you wrote as > examples. Hope that doesn''t muddy the waters though.I believe that I am gaining some insight into how this is all meant to pull together. One of the difficulties I face is that different people evidently have significantly different philosophies about how and what to test (surprise, surprise, surprise!). As I am coming at this from the pov of someone who has had no prior experience with this approach, and whose training in it to date has been entirely theoretical in nature, I find this somewhat confusing. At the moment I tend to see cucumber features as my sole method of testing, an approach that I realize is not favoured by many. I began learning cucumber by writing a set of fairly low level functional tests following the template generated by cucumber. Now, based on the advice I have received here, I am trying a much smaller scale approach and generating a few very high level features. However, in the back of my mind I still consider that eventually I will specify much of the implementation detail, model attributes, data normalization routines, input limits, and so forth as feature steps somewhere. Experience will eventually teach me whether that approach is sustainable or not. I presently have in mind two distinct types of tests/features that I wish to represent. The first is the end user type of feature writing which conforms generally to the analysis of system requirements. The second set of features will be more like functional tests, that exercise the implementation details. My expression of the second type of features, the functional tests, are probably what is generating the greatest controversy. Some practitioners evidently see BDD features more or less as a pure analytical tool. They expect that functional and unit tests will be conducted mostly in a "traditional" manner, via test unit or rspec or similar testing framework. With this approach there is no need, or desire, that features elaborate great detail regarding implementation since that is done elsewhere. The dichotomy between feature steps and step definitions is another point of confusion for me. Take your reference to "if internally I call out to steps more like the ones you wrote." Does this refer to feature "Steps" or to step definition "Steps". I suspect the latter. In that case one can imagine that step definitions become rather more elaborate structures. Some step definitions perhaps even assume the appearance and role of rspec specs and have only a remotely dependent relationship to any feature step. It is in these obscure details that much of my confusion arises. Perhaps I have inferred your meaning correctly, perhaps I have totally misunderstood it. Regardless of which is true, doubt remains. -- Posted via http://www.ruby-forum.com/.
James Byrne <lists at ruby-forum.com> writes:> Matt Wynne wrote: >> On 9 Dec 2008, at 19:06, James Byrne wrote: >> >>> should >>> >>> When /authentication by authlogic/ do >>> #TODO: Find out how to test for a loaded gem within an application >>> end >> >> If you''re working at the level of Cucumber features, I would stay well >> away from implementation details like the particular technology you''re >> using for authentication. >> > > The features for this simply say something along the lines of: > > Scenario: Application has known users > Given that the application has users > When the user signs on > Then the user is authenticatedWTBV? (Where''s the business value (I totally made that up, but I''m gonna stick with it :)) On its own, being authenticated doesn''t have any business value. Authentication only *enhances* business value. Consider a feature that lets me manage finances. That has some business value. Then we ensure that only authenticated users may manage their finances, which means we can charge people for the service - enhancing the business value that already exists.>> Remember this is called Behaviour Driven Development[1] for a reason. >> You''re focussing on specifying the desired *behaviour* of the system. >> How you chose to satisfy the user''s need for that behaviour is an >> entirely separate matter. > > Nonetheless, in the step definitions I must test the behaviour in a > fashion which first drives and then confirms the implementation > details.I don''t understand this. Your features/specs drive the implementation, but they confirm the *behavior*.> If the chosen implementation depends upon a third party plugin or gem > then surely one should test for its provision? What is the alternative? > Write tests to test that gems behaviour?I would specify my app''s behavior, not the gem''s. And I certainly wouldn''t test that the gem is there. If it''s not, then my code is going to blow up anyway! Instead, write features that specify *what* you want from your application. Given a user named ''padillac'' When I log in as ''padillac'' Then I should see ''Hello, padillac!'' And I should see ''Edit your account'' How you handle this - whether it''s restful_authentication, authlogic, or roll-your-own - doesn''t matter.>> A large benefit of having Acceptance Tests, IMO, is that you could >> radically change your implementation under the hood, such as switching >> to a different authentication mechanism, and the features would still >> be valid. You might have to code up some of your steps a little >> differently, but as long as the users want the same thing from the >> system, the features should not have to change in the slightest. >> > > If the implementation changed then the step definitions must change to > suit, must they not?It depends on the level at which you''ve written your step definitions. If you''ve written them to hit the outer-most interface, for example using webrat in a rails app, then no, you wouldn''t need to change the definitions. If you wrote them at a lower level, creating a user and adding his id to the session, then there''s a chance you would have to change them. The key is writing the features themselves at a level of abstraction that *completely* hides the implementation. That way you can change your code and step definitions freely. What you''re left with is a balancing act, deciding which implementation details can force changes to your step definitions. If you write the definitions at a low level, accessing the model directly, you''ll likely need to change them when the definitions change. On the other hand, if you write them at a higher level, there''s the real possibility that minor changes to your app code will necessitate changes to your step definitions. Consider the following step definition: When /^I log in as ''(.*)''$/ do |username| visits login_url fills_in ''Username'', :with => username fills_in ''Password'', :with => ''password'' end If you change the template to read ''Login'' instead of ''Username'' for the username field, the step definition is going to have to change. So you have to evaluate each piece of your app, figure out what''s likely to change, determine the upfront and maintenance costs of writing step definitions at varying levels of abstraction, and make a decision. Then be thankful that you''re not building a bridge, because even if you get it wrong, the cost of fixing your mistake is relatively cheap :)> Not all features are of concern to all users/roles. Why should some > users be denied the use of features and BDD as a tool for their own > concerns simply because they are writing the code? It seems to me an > artificial, and logically insupportable, distinction to make.I don''t understand what you mean by this. Which users are "denied the use of features and BDD?" Could you please elaborate for me?> When /user "(.*)" is authenticated/ do |u| > # We use Authlogic for authentication so just test that gem is loaded > assert defined?("Authlogic") > endWTBV?! :) Pat
Pat Maddox wrote:>> >> Scenario: Application has known users >> Given that the application has users >> When the user signs on >> Then the user is authenticated > > WTBV? > > (Where''s the business value (I totally made that up, but I''m gonna stick > with it :))I like it too. That said, regulatory compliance and financial security are two that come to mind. And, yes, I and my users consider that both represent real business value. I am not trying to put forth a design philosophy. I am trying to discover what techniques experienced BDD practitioners consider useful and what they do not. My ignorance may lead me into astoundingly poor conclusions at times but, I am willing to bear the resulting ridicule if in the process I learn that which I should have realized. Fortunately, ridicule has been noteworthy by its absence and the knowledge revealed by respondents quite illuminating. I am beginning to see that there exists a wide range of acceptable practices gathered under the rubric of BDD. At the beginning I acquired the idea that step definitions were tied to feature steps N:1. Now I gather, perhaps incorrectly, that step definitions might replace discrete tests/specifications in frameworks like TestUnit and RSpec without the need for a directly corresponding feature step, N:N where N>= 0Maybe that is my problem. Perhaps I am trying too hard to use features for everything. I really do not want to scatter my tests over hell''s half acre. I do not wish to have some tests under ./test, others under ./spec and still others under ./features. I do not want to have three different rake tasks to run all the tests. I do not want to integrate rcov results from one test suite with the results from another. I do not wish to expend time on integrating test suites with one another. So, given all of these desires, perhaps I am considering features in a way that is considerably less than optimal. On the other hand, given that the rspec and testunit matchers are available in step definitions, is there any harm in using step definitions in place of specs and unit tests? In the absence of any other method, one could imagine I suppose a feature step that simply invokes that portion of the step definitions test suite that is not tied otherwise to any other feature step. -- Posted via http://www.ruby-forum.com/.