I noticed that besides not_compliant_on to disable tests on specific Ruby implementations, there is also compliant_on to enable tests only on selected implementations. There are ~120 such instances. To make IronRuby compliant, we not only have to reduce the number of *_tags.txt files, we also have to add ":ir" to all uses of compliant_on. However, this seems backwards. Shouldn''t we try to remove most uses of compliant_on, and instead add tags for the implementations where the test does not work? There may be a few valid uses of compliant_on for tests which only work on, say, JRuby. However, only a handful of the occurrences are of this kind. The vast majoriy include :ruby which means it should be the standard. Should it be a goal to change compliant_on so that it does not allow :ruby? Thanks, Shri -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://rubyforge.org/pipermail/ironruby-core/attachments/20090107/318fdedb/attachment.html>
I can check with Brian, but my understanding is as follows. The spec''s are meant to be a complete packaged spec. Tag files are a convenience for using the spec''s as a regression test. I feel, and I would guess this to be Brian''s feelings, that when an implementation is complete (as in everything is implemented), there should not be a set of tag files for that implementation. The other goal of the spec''s is to be a spec. To that goal, you shouldn''t have to go outside of the spec''s to find out compliance information. Including the information into the file makes it easier to see what a specific implementation of Ruby supports. Keeping the code DRY in tests isn''t as important as conveying information in those tests. JD From: Shri Borde Sent: Wednesday, January 07, 2009 12:26 AM To: Jim Deville Cc: ironruby-core at rubyforge.org Subject: compliant_on(:ruby) in RubySpecs I noticed that besides not_compliant_on to disable tests on specific Ruby implementations, there is also compliant_on to enable tests only on selected implementations. There are ~120 such instances. To make IronRuby compliant, we not only have to reduce the number of *_tags.txt files, we also have to add ":ir" to all uses of compliant_on. However, this seems backwards. Shouldn''t we try to remove most uses of compliant_on, and instead add tags for the implementations where the test does not work? There may be a few valid uses of compliant_on for tests which only work on, say, JRuby. However, only a handful of the occurrences are of this kind. The vast majoriy include :ruby which means it should be the standard. Should it be a goal to change compliant_on so that it does not allow :ruby? Thanks, Shri -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://rubyforge.org/pipermail/ironruby-core/attachments/20090107/dd951031/attachment-0001.html>
I see the point of having all the information within the specs. It is useful to read a spec and to be able to know how other implementations behave (assuming the implementation is solid enough as to not cause a huge number of usages of not_compliant_on in the test code base). However, I think compliant_on(:ruby, :ir) for example should be replaced with not_compliant_on(:jruby, :rubinius). That is, only opt-out should be supported, not opt-in. Otherwise, other implementations may not run the test even if they support that functionality. From: Jim Deville Sent: Wednesday, January 07, 2009 10:57 AM To: Shri Borde Cc: ironruby-core at rubyforge.org Subject: RE: compliant_on(:ruby) in RubySpecs I can check with Brian, but my understanding is as follows. The spec''s are meant to be a complete packaged spec. Tag files are a convenience for using the spec''s as a regression test. I feel, and I would guess this to be Brian''s feelings, that when an implementation is complete (as in everything is implemented), there should not be a set of tag files for that implementation. The other goal of the spec''s is to be a spec. To that goal, you shouldn''t have to go outside of the spec''s to find out compliance information. Including the information into the file makes it easier to see what a specific implementation of Ruby supports. Keeping the code DRY in tests isn''t as important as conveying information in those tests. JD From: Shri Borde Sent: Wednesday, January 07, 2009 12:26 AM To: Jim Deville Cc: ironruby-core at rubyforge.org Subject: compliant_on(:ruby) in RubySpecs I noticed that besides not_compliant_on to disable tests on specific Ruby implementations, there is also compliant_on to enable tests only on selected implementations. There are ~120 such instances. To make IronRuby compliant, we not only have to reduce the number of *_tags.txt files, we also have to add ":ir" to all uses of compliant_on. However, this seems backwards. Shouldn''t we try to remove most uses of compliant_on, and instead add tags for the implementations where the test does not work? There may be a few valid uses of compliant_on for tests which only work on, say, JRuby. However, only a handful of the occurrences are of this kind. The vast majoriy include :ruby which means it should be the standard. Should it be a goal to change compliant_on so that it does not allow :ruby? Thanks, Shri -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://rubyforge.org/pipermail/ironruby-core/attachments/20090107/d9710506/attachment.html>
http://rubyspec.org/wiki/rubyspec/Guards If you scroll to the compliant portion of the page, you''ll see that they actually have different meanings, and they are based on the reference implementation (CRuby). I had forgotten this until just now. JD From: Shri Borde Sent: Wednesday, January 07, 2009 11:39 AM To: Jim Deville Cc: ironruby-core at rubyforge.org Subject: RE: compliant_on(:ruby) in RubySpecs I see the point of having all the information within the specs. It is useful to read a spec and to be able to know how other implementations behave (assuming the implementation is solid enough as to not cause a huge number of usages of not_compliant_on in the test code base). However, I think compliant_on(:ruby, :ir) for example should be replaced with not_compliant_on(:jruby, :rubinius). That is, only opt-out should be supported, not opt-in. Otherwise, other implementations may not run the test even if they support that functionality. From: Jim Deville Sent: Wednesday, January 07, 2009 10:57 AM To: Shri Borde Cc: ironruby-core at rubyforge.org Subject: RE: compliant_on(:ruby) in RubySpecs I can check with Brian, but my understanding is as follows. The spec''s are meant to be a complete packaged spec. Tag files are a convenience for using the spec''s as a regression test. I feel, and I would guess this to be Brian''s feelings, that when an implementation is complete (as in everything is implemented), there should not be a set of tag files for that implementation. The other goal of the spec''s is to be a spec. To that goal, you shouldn''t have to go outside of the spec''s to find out compliance information. Including the information into the file makes it easier to see what a specific implementation of Ruby supports. Keeping the code DRY in tests isn''t as important as conveying information in those tests. JD From: Shri Borde Sent: Wednesday, January 07, 2009 12:26 AM To: Jim Deville Cc: ironruby-core at rubyforge.org Subject: compliant_on(:ruby) in RubySpecs I noticed that besides not_compliant_on to disable tests on specific Ruby implementations, there is also compliant_on to enable tests only on selected implementations. There are ~120 such instances. To make IronRuby compliant, we not only have to reduce the number of *_tags.txt files, we also have to add ":ir" to all uses of compliant_on. However, this seems backwards. Shouldn''t we try to remove most uses of compliant_on, and instead add tags for the implementations where the test does not work? There may be a few valid uses of compliant_on for tests which only work on, say, JRuby. However, only a handful of the occurrences are of this kind. The vast majoriy include :ruby which means it should be the standard. Should it be a goal to change compliant_on so that it does not allow :ruby? Thanks, Shri -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://rubyforge.org/pipermail/ironruby-core/attachments/20090107/7aeef882/attachment-0001.html>