I''m not sure how much test data I should be using in my specs.
I''m
writing specs for the Property model in my Rails app. Its "address"
attribute is going to be validated with this regex:
/\A\d+[a-z]? [-'', a-z]{2,128}\Z/i
At the moment, my plan is to spec out the following possibilities. A
property is invalid if its address:
1) doesn''t begin with a digit;
2) is shorter than 2 characters;
3) is longer than 128 characters;
I should also test that certain characters, such as ! @ # $ etc,
invalidate a property.
However, all of that seems like blacklisting, and achieves poor
coverage of the regular expression.
Should I create a list of valid and invalid addresses to test against
the regex? That seems like a decent idea, but seems synonymous with
fixtures. What would you guys recommend?
Cheers,
Nick
On Mon, Aug 25, 2008 at 12:53 PM, Nick Hoffman <nick at deadorange.com> wrote:> I''m not sure how much test data I should be using in my specs. I''m writing > specs for the Property model in my Rails app. Its "address" attribute is > going to be validated with this regex: > /\A\d+[a-z]? [-'', a-z]{2,128}\Z/i > > At the moment, my plan is to spec out the following possibilities. A > property is invalid if its address: > 1) doesn''t begin with a digit; > 2) is shorter than 2 characters; > 3) is longer than 128 characters; > > I should also test that certain characters, such as ! @ # $ etc, invalidate > a property. > > However, all of that seems like blacklisting, and achieves poor coverage of > the regular expression. > > Should I create a list of valid and invalid addresses to test against the > regex? That seems like a decent idea, but seems synonymous with fixtures. > What would you guys recommend? >I might do something like the following... describe Property, "email validations" do ["1invalid.email at example.com", "can''t start with a digit", "invalid.email2 at example.com", "can''t end with a digit" ].in_groups_of(2) do |email, description| it description do prop = Property.new :email => email prop.should_not be_valid prop.should have(1).error_on(:email) end end end -- Zach Dennis http://www.continuousthinking.com http://www.mutuallyhuman.com
On 2008-08-25, at 13:29, Zach Dennis wrote:> I might do something like the following... > > describe Property, "email validations" do > > ["1invalid.email at example.com", "can''t start with a digit", > "invalid.email2 at example.com", "can''t end with a digit" > ].in_groups_of(2) do |email, description| > it description do > prop = Property.new :email => email > prop.should_not be_valid > prop.should have(1).error_on(:email) > end > end > end > > > -- > Zach Dennis > http://www.continuousthinking.com > http://www.mutuallyhuman.comHi Zach. That''s a great way of iterating over test data. Do you have any suggestions for how much test data to use? Cheers, Nick
On Mon, Aug 25, 2008 at 2:50 PM, Nick Hoffman <nick at deadorange.com> wrote:> On 2008-08-25, at 13:29, Zach Dennis wrote: >> >> I might do something like the following... >> >> describe Property, "email validations" do >> >> ["1invalid.email at example.com", "can''t start with a digit", >> "invalid.email2 at example.com", "can''t end with a digit" >> ].in_groups_of(2) do |email, description| >> it description do >> prop = Property.new :email => email >> prop.should_not be_valid >> prop.should have(1).error_on(:email) >> end >> end >> end >> >> >> -- Zach Dennis >> http://www.continuousthinking.com >> http://www.mutuallyhuman.com > > Hi Zach. That''s a great way of iterating over test data. Do you have any > suggestions for how much test data to use? >I''d probably start with the first invalid email/description that I can think of. Perhaps it can''t start with a digit. Then I''d update the regexp to reflect that. Next I''d add a second invalid email/description and update the regexp to reflect that. Rinse and repeat until your regexp is where you need it. This should help you not add redundant emails which test the same thing. When you get to the case where you allow from 2 up to 128 characters, you could just test those two edge cases (rather than emails which 2 characters, 3 characters, 4 characters, ... 128 characters. -- Zach Dennis http://www.continuousthinking.com http://www.mutuallyhuman.com
On Mon, Aug 25, 2008 at 3:02 PM, Zach Dennis <zach.dennis at gmail.com> wrote:> On Mon, Aug 25, 2008 at 2:50 PM, Nick Hoffman <nick at deadorange.com> wrote: >> On 2008-08-25, at 13:29, Zach Dennis wrote: >>> >>> I might do something like the following... >>> >>> describe Property, "email validations" do >>> >>> ["1invalid.email at example.com", "can''t start with a digit", >>> "invalid.email2 at example.com", "can''t end with a digit" >>> ].in_groups_of(2) do |email, description| >>> it description do >>> prop = Property.new :email => email >>> prop.should_not be_valid >>> prop.should have(1).error_on(:email) >>> end >>> end >>> end >>> >>> >>> -- Zach Dennis >>> http://www.continuousthinking.com >>> http://www.mutuallyhuman.com >> >> Hi Zach. That''s a great way of iterating over test data. Do you have any >> suggestions for how much test data to use? >> > > I''d probably start with the first invalid email/description that I can > think of. Perhaps it can''t start with a digit. Then I''d update the > regexp to reflect that. Next I''d add a second invalid > email/description and update the regexp to reflect that. Rinse and > repeat until your regexp is where you need it.This is nit-picky, I realize, but saying "until your regexp is where you need it" suggests that you''ll know when you get there by looking at the regexp. I''d suggest that you know when you get there by looking at the code examples and feeling confident that you''ve covered all of the edge cases.> This should help you > not add redundant emails which test the same thing. When you get to > the case where you allow from 2 up to 128 characters, you could just > test those two edge cases (rather than emails which 2 characters, 3 > characters, 4 characters, ... 128 characters.If the boundaries are 2 and 128, I would test 1,2,3,127,128,129 - the boundary and either side of it. Might seem like overkill, but I''ve seen a bug or two make its way to production because the boundary wasn''t clearly understood. FWIW, David> > -- > Zach Dennis > http://www.continuousthinking.com > http://www.mutuallyhuman.com > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users >
Hi all, On 25 Aug 2008, at 17:53, Nick Hoffman wrote:> At the moment, my plan is to spec out the following possibilities. > A property is invalid if its address: > 1) doesn''t begin with a digit; > 2) is shorter than 2 characters; > 3) is longer than 128 characters;Personally I think that if those three points are your specification then your RSpec should match that. So: * it "should not begin with a digit", * it "should not be shorter than 2 characters", * it "should not be longer than 128 characters" and * it "should allow valid addresses" (this is for clarity, so you have an example of a valid address in your documentation). If your regex allows or disallows more or less than those four clauses then your specification does not match your implementation. Baz Rahoul Baruah Web design and development: http://www.3hv.co.uk/ Nottingham Forest: http://www.eighteensixtyfive.co.uk/ Serious Rails Hosting: http://www.brightbox.co.uk/ Lifecast: http://www.madeofstone.net/ -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://rubyforge.org/pipermail/rspec-users/attachments/20080827/9e8d3040/attachment-0001.html>
> > On 25 Aug 2008, at 17:53, Nick Hoffman wrote: > >> At the moment, my plan is to spec out the following possibilities. >> A property is invalid if its address: >> 1) doesn''t begin with a digit; >> 2) is shorter than 2 characters; >> 3) is longer than 128 characters; > > > > Personally I think that if those three points are your > specification then your RSpec should match that. > > So: > * it "should not begin with a digit", > * it "should not be shorter than 2 characters", > * it "should not be longer than 128 characters" and > * it "should allow valid addresses" (this is for clarity, so you > have an example of a valid address in your documentation). > > If your regex allows or disallows more or less than those four > clauses then your specification does not match your implementation. >So this is interesting, because my approach seems to be more like this: describe "when the address does not begin with a digit" do it "should be invalid" do ... end end describe "when the address is less than two characters long" do it "should be invalid" do ... end end describe "when the address is more than two characters long" do describe "and the address is less than 128 characters long" do it "should be valid" do ... end end describe "and the address is more than 128 characters long" do it "should be invalid" do ... end end end Is anyone else using ExampleGroups like this? Or is it just me?! cheers, Matt -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://rubyforge.org/pipermail/rspec-users/attachments/20080827/e9f7005e/attachment.html>
On 27 Aug 2008, at 16:35, Matt Wynne wrote:> describe "when the address is less than two characters long" do > it "should be invalid" do ... end > end > > describe "when the address is more than two characters long" do > > describe "and the address is less than 128 characters long" do > it "should be valid" do ... end > end > > describe "and the address is more than 128 characters long" do > it "should be invalid" do ... end > end > > endFor this, trivial, example, I find that far too verbose. The "specification" says "if it is less than 2 characters then invalid" and "if it is more than 128 characters then invalid" . But you are actually running three checks - "if less than 2", "if greater than 2 but less than 128" and "if greater than 2 and greater than 128". B Rahoul Baruah Web design and development: http://www.3hv.co.uk/ Nottingham Forest: http://www.eighteensixtyfive.co.uk/ Serious Rails Hosting: http://www.brightbox.co.uk/ Lifecast: http://www.madeofstone.net/ -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://rubyforge.org/pipermail/rspec-users/attachments/20080827/2fd03ded/attachment.html>
On 2008-08-27, at 12:57, Rahoul Baruah wrote:> For this, trivial, example, I find that far too verbose. > > The "specification" says "if it is less than 2 characters then > invalid" and "if it is more than 128 characters then invalid" . > > But you are actually running three checks - "if less than 2", "if > greater than 2 but less than 128" and "if greater than 2 and greater > than 128". > > B > > > Rahoul BaruahThis is what I ended up with: Property addresses that are valid - can''t be 129 characters - can''t have a 1-letter street name - can''t have an & - can''t have an ! - can''t have a " - can''t have 2 letters after the street number - can''t be missing a street number - can''t have only 3 characters - can''t be empty - can have 128 characters - can have 127 characters - can have only 5 characters - can have only 4 characters - can have a , - can have a - - can have a '' - can have 1 letter after the street number - can have multiple spaces Each of the "can ..." examples are proper addresses in the sense that they''re variations of: - "123 A Street With A Long Name" - "123B Maple Ave" - "123 O''Connor Street" - etc One thing that I didn''t do is write examples for each invalid character. That''d be too hairy and verbose. Instead, I picked a few, and wrote examples for them, as you can see above (Eg: "can''t have an &"). Cheers, Nick
On Wed, Aug 27, 2008 at 1:03 PM, Nick Hoffman <nick at deadorange.com> wrote:> On 2008-08-27, at 12:57, Rahoul Baruah wrote: > >> For this, trivial, example, I find that far too verbose. >> >> The "specification" says "if it is less than 2 characters then invalid" >> and "if it is more than 128 characters then invalid" . >> >> But you are actually running three checks - "if less than 2", "if greater >> than 2 but less than 128" and "if greater than 2 and greater than 128". >> >> B >> >> >> Rahoul Baruah >> > > This is what I ended up with: > > Property addresses that are valid > - can''t be 129 characters > - can''t have a 1-letter street name > - can''t have an & > - can''t have an ! > - can''t have a " > - can''t have 2 letters after the street number > - can''t be missing a street number > - can''t have only 3 characters > - can''t be empty > - can have 128 characters > - can have 127 characters > - can have only 5 characters > - can have only 4 characters > - can have a , > - can have a - > - can have a '' > - can have 1 letter after the street number > - can have multiple spaces > > Each of the "can ..." examples are proper addresses in the sense that > they''re variations of: > - "123 A Street With A Long Name" > - "123B Maple Ave" > - "123 O''Connor Street" > - etc > > One thing that I didn''t do is write examples for each invalid character. > That''d be too hairy and verbose. Instead, I picked a few, and wrote examples > for them, as you can see above (Eg: "can''t have an &").If you want to (not necessarily advising this, but I''ve seen it done) you can do this: [''!'',''('','')'',''&''].each do |char| it "can''t have #{char}" do ... end end That makes the output very verbose, but the spec file is easy to grok.> > > Cheers, > Nick > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://rubyforge.org/pipermail/rspec-users/attachments/20080827/dccbbdfb/attachment.html>
what if my office were at Route 102 & Yahoo! Way, Suite #123 :) On Aug 27, 2008, at 2:03 PM, Nick Hoffman wrote:> On 2008-08-27, at 12:57, Rahoul Baruah wrote: >> For this, trivial, example, I find that far too verbose. >> >> The "specification" says "if it is less than 2 characters then >> invalid" and "if it is more than 128 characters then invalid" . >> >> But you are actually running three checks - "if less than 2", "if >> greater than 2 but less than 128" and "if greater than 2 and >> greater than 128". >> >> B >> >> >> Rahoul Baruah > > This is what I ended up with: > > Property addresses that are valid > - can''t be 129 characters > - can''t have a 1-letter street name > - can''t have an & > - can''t have an ! > - can''t have a " > - can''t have 2 letters after the street number > - can''t be missing a street number > - can''t have only 3 characters > - can''t be empty > - can have 128 characters > - can have 127 characters > - can have only 5 characters > - can have only 4 characters > - can have a , > - can have a - > - can have a '' > - can have 1 letter after the street number > - can have multiple spaces > > Each of the "can ..." examples are proper addresses in the sense > that they''re variations of: > - "123 A Street With A Long Name" > - "123B Maple Ave" > - "123 O''Connor Street" > - etc > > One thing that I didn''t do is write examples for each invalid > character. That''d be too hairy and verbose. Instead, I picked a > few, and wrote examples for them, as you can see above (Eg: "can''t > have an &"). > > Cheers, > Nick > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users
On 2008-08-27, at 14:55, Jonathan Linowes wrote:> what if my office were at > > Route 102 & Yahoo! Way, Suite #123 > > :)Good point. Thinking about it again, there''s probably not all that much need to prevent addresses from having symbols such as !, &, ", etc. If people really want to put them in, why not? Does your office actually not have a proper street number? -Nick
We''d recommend that you start sending our your resum?. :) cr On Aug 27, 2008, at 1:55 PM, Jonathan Linowes wrote:> what if my office were at > > Route 102 & Yahoo! Way, Suite #123 > > :) > > > On Aug 27, 2008, at 2:03 PM, Nick Hoffman wrote: > >> On 2008-08-27, at 12:57, Rahoul Baruah wrote: >>> For this, trivial, example, I find that far too verbose. >>> >>> The "specification" says "if it is less than 2 characters then >>> invalid" and "if it is more than 128 characters then invalid" . >>> >>> But you are actually running three checks - "if less than 2", "if >>> greater than 2 but less than 128" and "if greater than 2 and >>> greater than 128". >>> >>> B >>> >>> >>> Rahoul Baruah >> >> This is what I ended up with: >> >> Property addresses that are valid >> - can''t be 129 characters >> - can''t have a 1-letter street name >> - can''t have an & >> - can''t have an ! >> - can''t have a " >> - can''t have 2 letters after the street number >> - can''t be missing a street number >> - can''t have only 3 characters >> - can''t be empty >> - can have 128 characters >> - can have 127 characters >> - can have only 5 characters >> - can have only 4 characters >> - can have a , >> - can have a - >> - can have a '' >> - can have 1 letter after the street number >> - can have multiple spaces
On 2008-08-27, at 14:46, David Chelimsky wrote:> If you want to (not necessarily advising this, but I''ve seen it > done) you can do this: > > [''!'',''('','')'',''&''].each do |char| > it "can''t have #{char}" do > ... > end > end > > That makes the output very verbose, but the spec file is easy to grok.Hi David. I ended up creating valid and invalid test data, as well as a method to iterate over them and write the descriptions and examples for me. You can have a look at it at http://pastie.org/261175 . If you have any suggestions for improvements, I''d love to hear them. Cheers, Nick
On 2008-08-27, at 15:24, Nick Hoffman wrote:> Hi David. I ended up creating valid and invalid test data, as well > as a method to iterate over them and write the descriptions and > examples for me. You can have a look at it at http://pastie.org/ > 261175 . If you have any suggestions for improvements, I''d love to > hear them. > > Cheers, > NickI forgot to mention that I''m using the "factories-and-workers" plugin: http://github.com/dfl/factories-and-workers/ which provides the the "build_property" method.
On Wed, Aug 27, 2008 at 12:05 PM, Nick Hoffman <nick at deadorange.com> wrote:> On 2008-08-27, at 14:55, Jonathan Linowes wrote: > >> what if my office were at >> >> Route 102 & Yahoo! Way, Suite #123 >> >> :) >> > > Good point. Thinking about it again, there''s probably not all that much > need to prevent addresses from having symbols such as !, &, ", etc. If > people really want to put them in, why not? >Validation isn''t just about preventing users from doing things they want to do - it''s also to prevent errors. At any rate, what consitutes a valid address is a business rule, and isn''t a programming issue. ///ark -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://rubyforge.org/pipermail/rspec-users/attachments/20080827/19c7c0ba/attachment.html>
On Wed, Aug 27, 2008 at 2:24 PM, Nick Hoffman <nick at deadorange.com> wrote:> On 2008-08-27, at 14:46, David Chelimsky wrote: >> >> If you want to (not necessarily advising this, but I''ve seen it done) you >> can do this: >> >> [''!'',''('','')'',''&''].each do |char| >> it "can''t have #{char}" do >> ... >> end >> end >> >> That makes the output very verbose, but the spec file is easy to grok. > > Hi David. I ended up creating valid and invalid test data, as well as a > method to iterate over them and write the descriptions and examples for me. > You can have a look at it at http://pastie.org/261175 . If you have any > suggestions for improvements, I''d love to hear them.Hey Nick, I can definitely see some value in this, though there are a few things that I''d do differently. Have a look at http://pastie.org/261642 and let me know if you have any questions/comments. The only thing I''m not sure about is the actual method name - it''s not speaking to me, but I''m at a loss for a better one. Maybe after some coffee ... Cheers, David> > Cheers, > Nick
On 2008-08-28, at 08:31, David Chelimsky wrote:> On Wed, Aug 27, 2008 at 2:24 PM, Nick Hoffman <nick at deadorange.com> > wrote: >> On 2008-08-27, at 14:46, David Chelimsky wrote: >>> >>> If you want to (not necessarily advising this, but I''ve seen it >>> done) you >>> can do this: >>> >>> [''!'',''('','')'',''&''].each do |char| >>> it "can''t have #{char}" do >>> ... >>> end >>> end >>> >>> That makes the output very verbose, but the spec file is easy to >>> grok. >> >> Hi David. I ended up creating valid and invalid test data, as well >> as a >> method to iterate over them and write the descriptions and examples >> for me. >> You can have a look at it at http://pastie.org/261175 . If you have >> any >> suggestions for improvements, I''d love to hear them. > > Hey Nick, > > I can definitely see some value in this, though there are a few things > that I''d do differently. Have a look at http://pastie.org/261642 and > let me know if you have any questions/comments. The only thing I''m not > sure about is the actual method name - it''s not speaking to me, but > I''m at a loss for a better one. Maybe after some coffee ... > > Cheers, > DavidThanks for taking a look and making some suggestions, David. Much appreciated. I''ve incorporated some of them into the method. The method''s name was "describe_property_attributes" because it was being used to describe the Property (as in "real estate") model in my application. I''ve just finished abstracting it out so that it can now be used with any model class. If you''re interested, I''ve pasted it at http://pastie.org/261829
I started using a set of methods to deal with this problem from code in the RadiantCMS(http://radiantcms.org/) project. http://gist.github.com/7936 I stopped using it after a while finding the tests did not read well. Your method improves on Radiant''s which has tempted me to start testing models like this again. Thanks! -- Joseph Wilk http://www.joesniff.co.uk Nick Hoffman wrote:> On 2008-08-28, at 08:31, David Chelimsky wrote: >>>> ... >>> You can have a look at it at http://pastie.org/261175 . If you have >> >> Cheers, >> David > > Thanks for taking a look and making some suggestions, David. Much > appreciated. I''ve incorporated some of them into the method. > > The method''s name was "describe_property_attributes" because it was > being used to describe the Property (as in "real estate") model in my > application. I''ve just finished abstracting it out so that it can now > be used with any model class. If you''re interested, I''ve pasted it at > http://pastie.org/261829-- Posted via http://www.ruby-forum.com/.
On 2008-08-29, at 04:51, Joseph Wilk wrote:> I started using a set of methods to deal with this problem from code > in > the RadiantCMS(http://radiantcms.org/) project. > > http://gist.github.com/7936 > > I stopped using it after a while finding the tests did not read well. > Your method improves on Radiant''s which has tempted me to start > testing > models like this again. > > Thanks! > -- > Joseph Wilk > http://www.joesniff.co.ukThanks for the vote of confidence, Joseph! I have two other methods that go alongside #describe_model_attribute which you might find useful. I''m in the process of wrapping them up into a plugin. Until that''s finished, I''ve pasted them at http://pastie.org/262508 and http://pastie.org/262509 . Cheers, Nick
On Fri, Aug 29, 2008 at 9:03 AM, Nick Hoffman <nick at deadorange.com> wrote:> On 2008-08-29, at 04:51, Joseph Wilk wrote: >> >> I started using a set of methods to deal with this problem from code in >> the RadiantCMS(http://radiantcms.org/) project. >> >> http://gist.github.com/7936 >> >> I stopped using it after a while finding the tests did not read well. >> Your method improves on Radiant''s which has tempted me to start testing >> models like this again. >> >> Thanks! >> -- >> Joseph Wilk >> http://www.joesniff.co.uk > > Thanks for the vote of confidence, Joseph! I have two other methods that go > alongside #describe_model_attribute which you might find useful. I''m in the > process of wrapping them up into a plugin. Until that''s finished, I''ve > pasted them at http://pastie.org/262508 and http://pastie.org/262509 .Hey Nick - I''m not clear why you need these, given that you already have the describe_model_attributes macro, which you can use to handle nils and will give you a meaningful error if there is a problem with your factory. What''s your goal here?> > Cheers, > Nick > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users >
On 2008-08-29, at 11:05, David Chelimsky wrote:> On Fri, Aug 29, 2008 at 9:03 AM, Nick Hoffman <nick at deadorange.com> > wrote: >> On 2008-08-29, at 04:51, Joseph Wilk wrote: >>> >>> I started using a set of methods to deal with this problem from >>> code in >>> the RadiantCMS(http://radiantcms.org/) project. >>> >>> http://gist.github.com/7936 >>> >>> I stopped using it after a while finding the tests did not read >>> well. >>> Your method improves on Radiant''s which has tempted me to start >>> testing >>> models like this again. >>> >>> Thanks! >>> -- >>> Joseph Wilk >>> http://www.joesniff.co.uk >> >> Thanks for the vote of confidence, Joseph! I have two other methods >> that go >> alongside #describe_model_attribute which you might find useful. >> I''m in the >> process of wrapping them up into a plugin. Until that''s finished, >> I''ve >> pasted them at http://pastie.org/262508 and http://pastie.org/ >> 262509 . > > Hey Nick - I''m not clear why you need these, given that you already > have the describe_model_attributes macro, which you can use to handle > nils and will give you a meaningful error if there is a problem with > your factory. What''s your goal here?Hi David. My goal with these methods is to automate the speccing of model attributes and reduce the amount of code that needs to be written. I created #describe_model_factory so that there would be a spec specifically for the factories that I create, rather than relying on other specs to indirectly ensure that my factories work properly. Looking at the methods again, you''re correct that #describe_model_attribute includes most of the functionality of #check_model_attributes_when_nil . That''s not very DRY then, eh? I should probably get rid of #check_model_attributes_when_nil , and add an argument to #describe_model_attribute that allows the user to specify whether or not an error message should exist when the attribute being specced is invalid. How does that sound? -Nick