I know that the issue of closing tickets was brought up a couple of days ago and I''m hoping that others might be amniable to a suggestion regarding how tickets should be closed. To me the best way to prepare a ticket for closing is to actually either a.) write a unit test which demonstrates that the ticket is not applicable or b.) point to a unit test which does the same. To just close a report with "works for me" or with "won''t fix" seems to be dangerous at best, especially when it is a bug report. By going through the process of creating a unit test before closing we get the benefit of proof, plus the framework gets better test coverage and the test can provide the basis for additional tests in the future if more information is provided for the issue. Any thoughts on this? Does this seem like a reasonable thing to do? V/r Anthony Eden -- Cell: 808 782-5046 Current Location: Melbourne, FL --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To post to this group, send email to rubyonrails-core@googlegroups.com To unsubscribe from this group, send email to rubyonrails-core-unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/rubyonrails-core?hl=en -~----------~----~----~----~------~----~------~--~---
On 24/01/07, Anthony Eden <anthonyeden@gmail.com> wrote:> > I know that the issue of closing tickets was brought up a couple of > days ago and I''m hoping that others might be amniable to a suggestion > regarding how tickets should be closed. > > To me the best way to prepare a ticket for closing is to actually > either a.) write a unit test which demonstrates that the ticket is not > applicable> or b.) point to a unit test which does the same. To just > close a report with "works for me" or with "won''t fix" seems to be > dangerous at best, especially when it is a bug report. By going > through the process of creating a unit test before closing we get the > benefit of proof, plus the framework gets better test coverage and the > test can provide the basis for additional tests in the future if more > information is provided for the issue. > > Any thoughts on this? Does this seem like a reasonable thing to do?I don''t see how a test can be written to show something won''t be fixed. I also don''t how a test can prove an issue doesn''t exist. If tickets are closed with ''works for me'', the closer couldn''t think of a test that fails. It doesn''t prove the issue doesn''t exist, but then, proving a negative isn''t exactly easy. The onus should be on those with an interest in reopening the ticket to provide a failing test, rather than on those closing tickets to write passing tests. Tom --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To post to this group, send email to rubyonrails-core@googlegroups.com To unsubscribe from this group, send email to rubyonrails-core-unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/rubyonrails-core?hl=en -~----------~----~----~----~------~----~------~--~---
On 1/24/07, Tom Ward <tom@popdog.net> wrote:> > On 24/01/07, Anthony Eden <anthonyeden@gmail.com> wrote: > > > > I know that the issue of closing tickets was brought up a couple of > > days ago and I''m hoping that others might be amniable to a suggestion > > regarding how tickets should be closed. > > > > To me the best way to prepare a ticket for closing is to actually > > either a.) write a unit test which demonstrates that the ticket is not > > applicable > > > or b.) point to a unit test which does the same. To just > > close a report with "works for me" or with "won''t fix" seems to be > > dangerous at best, especially when it is a bug report. By going > > through the process of creating a unit test before closing we get the > > benefit of proof, plus the framework gets better test coverage and the > > test can provide the basis for additional tests in the future if more > > information is provided for the issue. > > > > Any thoughts on this? Does this seem like a reasonable thing to do? > > I don''t see how a test can be written to show something won''t be > fixed. I also don''t how a test can prove an issue doesn''t exist. > > If tickets are closed with ''works for me'', the closer couldn''t think > of a test that fails. It doesn''t prove the issue doesn''t exist, but > then, proving a negative isn''t exactly easy. The onus should be on > those with an interest in reopening the ticket to provide a failing > test, rather than on those closing tickets to write passing tests. > > TomAgreed - if you don''t provide a failing test case with a defect, or at least a very detailed bug report, don''t be surprised if it gets closed rather quickly. Thats how open source works - the burden is on the reporter to prove the bug exists as best as possible. - rob --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To post to this group, send email to rubyonrails-core@googlegroups.com To unsubscribe from this group, send email to rubyonrails-core-unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/rubyonrails-core?hl=en -~----------~----~----~----~------~----~------~--~---
On 1/24/07, Tom Ward <tom@popdog.net> wrote:> > I don''t see how a test can be written to show something won''t be > fixed."Wontfix" resolution is usually for requests for specific features be changed or enhanced. Real bugs should never be resolved wontfix because they are always fixed. I also don''t how a test can prove an issue doesn''t exist. Easy - take what the reported has described and turn it into a test. If it passes it is proof that it doesn''t exist, and you''ve raised test coverage a tiny bit. If tickets are closed with ''works for me'', the closer couldn''t think> of a test that fails. It doesn''t prove the issue doesn''t exist, but > then, proving a negative isn''t exactly easy. The onus should be on > those with an interest in reopening the ticket to provide a failing > test, rather than on those closing tickets to write passing tests.I agree with this. About defects that are reported with less than a minimal clue how to reproduce (and without a failing tests, obviously) - should they be resolved as "untested", or that resolution only applies to ready patches? --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To post to this group, send email to rubyonrails-core@googlegroups.com To unsubscribe from this group, send email to rubyonrails-core-unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/rubyonrails-core?hl=en -~----------~----~----~----~------~----~------~--~---
On Jan 24, 2007, at 17:51, Anthony Eden wrote:> [...] Any thoughts on this? Does this seem like a reasonable thing > to do?Why can''t we assume the reporter will come back to re-open the ticket if he or she feels the ticket should not have been closed? Can you give an example of a non-trivial issue that has not been fixed just because someone closed the ticket for the wrong reason? Kind regards, Thijs van der Vossen -- Fingertips - http://www.fngtps.com Phone: +31 (0)6 24204845 Skype: tvandervossen MSN Messenger: thijs@fngtps.com iChat/AOL: t.vandervossen@mac.com Jabber IM: thijs@jabber.org --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To post to this group, send email to rubyonrails-core@googlegroups.com To unsubscribe from this group, send email to rubyonrails-core-unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/rubyonrails-core?hl=en -~----------~----~----~----~------~----~------~--~---
On 1/24/07, Tom Ward <tom@popdog.net> wrote:> > I don''t see how a test can be written to show something won''t be > fixed. I also don''t how a test can prove an issue doesn''t exist.A passing test case which demonstrates that specific input results in specific output indicates that an issue doesn''t exist, at least using the assumptions of the original issue. Granted a test case may pass in one case and fail in another due to variations in the environment (OSes, configurations, etc) however a test is better than no test. As for test cases for something which won''t be fixed, at least point to or provide a test case which demonstrates the expected behavior. If there is no test case then the correct behavior is a matter of interpretation and will often lead to confusion, bugs, more people raising issues, etc.> If tickets are closed with ''works for me'', the closer couldn''t think > of a test that fails. It doesn''t prove the issue doesn''t exist, but > then, proving a negative isn''t exactly easy. The onus should be on > those with an interest in reopening the ticket to provide a failing > test, rather than on those closing tickets to write passing tests.How do you know that the closer couldn''t think of a test that fails? Unless the closer creates a test that passes based on a set of assumptions then how do you know what lead them to believe that it works for them? We don''t know what the closer''s assumptions were, and the point of test cases is to define something which can be applied repeatedly with the same assumptions and which results in the same outcome. I think if an issue can be legitimately closed with a passing test case then the onus does reasonably shift to the original person who wrote the ticket to provide a test case that fails. V/r Anthony Eden -- Cell: 808 782-5046 Current Location: Melbourne, FL --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To post to this group, send email to rubyonrails-core@googlegroups.com To unsubscribe from this group, send email to rubyonrails-core-unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/rubyonrails-core?hl=en -~----------~----~----~----~------~----~------~--~---
On 1/24/07, Rob Sanheim <rsanheim@gmail.com> wrote:> > Agreed - if you don''t provide a failing test case with a defect, or at > least a very detailed bug report, don''t be surprised if it gets closed > rather quickly. Thats how open source works - the burden is on the > reporter to prove the bug exists as best as possible. > > - robAt least close it with untested if this is the case. Closing with wontfix or worksforme is not the appropriate response, IMO. V/r Anthony Eden -- Cell: 808 782-5046 Current Location: Melbourne, FL --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To post to this group, send email to rubyonrails-core@googlegroups.com To unsubscribe from this group, send email to rubyonrails-core-unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/rubyonrails-core?hl=en -~----------~----~----~----~------~----~------~--~---
Non-trivial is in the eye of the beholder, however here are some examples of issues which have been closed in the last few days as wontfix or worksforme, but which do not provide a test to verify. http://dev.rubyonrails.org/ticket/2851 http://dev.rubyonrails.org/ticket/4415 http://dev.rubyonrails.org/ticket/6572 http://dev.rubyonrails.org/ticket/5455 http://dev.rubyonrails.org/ticket/5688 In all of these cases the closure is based on someone stating that they do not see this behavior, or that they have tested it, but there is no way to verify their tests. All I''m suggesting is that by providing a test case anyone would be able to prove it on their own environment, including the original author of the issue should they still be experiencing the problem. V/r Anthony Eden On 1/24/07, Thijs van der Vossen <t.vandervossen@gmail.com> wrote:> > On Jan 24, 2007, at 17:51, Anthony Eden wrote: > > [...] Any thoughts on this? Does this seem like a reasonable thing > > to do? > > Why can''t we assume the reporter will come back to re-open the ticket > if he or she feels the ticket should not have been closed? > > Can you give an example of a non-trivial issue that has not been > fixed just because someone closed the ticket for the wrong reason? > > Kind regards, > Thijs van der Vossen > > -- > Fingertips - http://www.fngtps.com > > Phone: +31 (0)6 24204845 > Skype: tvandervossen > > MSN Messenger: thijs@fngtps.com > iChat/AOL: t.vandervossen@mac.com > Jabber IM: thijs@jabber.org > > > > > > >-- Cell: 808 782-5046 Current Location: Melbourne, FL --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To post to this group, send email to rubyonrails-core@googlegroups.com To unsubscribe from this group, send email to rubyonrails-core-unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/rubyonrails-core?hl=en -~----------~----~----~----~------~----~------~--~---
Anthony Eden wrote:> On 1/24/07, Tom Ward <tom@popdog.net> wrote: >> I don''t see how a test can be written to show something won''t be >> fixed. I also don''t how a test can prove an issue doesn''t exist. > > A passing test case which demonstrates that specific input results in > specific output indicates that an issue doesn''t exist, at least using > the assumptions of the original issue. Granted a test case may pass in > one case and fail in another due to variations in the environment > (OSes, configurations, etc) however a test is better than no test. As > for test cases for something which won''t be fixed, at least point to > or provide a test case which demonstrates the expected behavior. If > there is no test case then the correct behavior is a matter of > interpretation and will often lead to confusion, bugs, more people > raising issues, etc. > >> If tickets are closed with ''works for me'', the closer couldn''t think >> of a test that fails. It doesn''t prove the issue doesn''t exist, but >> then, proving a negative isn''t exactly easy. The onus should be on >> those with an interest in reopening the ticket to provide a failing >> test, rather than on those closing tickets to write passing tests. > > How do you know that the closer couldn''t think of a test that fails? > Unless the closer creates a test that passes based on a set of > assumptions then how do you know what lead them to believe that it > works for them? We don''t know what the closer''s assumptions were, and > the point of test cases is to define something which can be applied > repeatedly with the same assumptions and which results in the same > outcome. > > I think if an issue can be legitimately closed with a passing test > case then the onus does reasonably shift to the original person who > wrote the ticket to provide a test case that fails.You are right to point out that variations in environment can cause a test to fail for the creator of a ticket, but succeed when tested by a Rails committer. But the situation you are concerned about seems to be when the ticket creator hasn''t given a failing (for him/her) test. You are proposing that the Rails committer who wants to close a ticket (i) interprets the reported problem in terms of a test, (ii) writes the test, and (iii) demonstrates that, in some ''standard'' environment, the test passes. In other words, you are proposing that Rails is presumed guilty until proved innocent, and asking the committers to act as the lawyers for the defendant. The number of Rails users is in the tens or hundreds of thousands, with a wide variety of levels of skill and experience. You can see from reading the mailing list how often people raise ''problems'' that turn out to be the result of their own error or misunderstanding. The number of committers is small, and they are highly skilled. In the circumstances, the committers *must* be able to close tickets without spending a lot of time explaining why, and the burden must then fall on the ticket creator to justify reopening the ticket. Anything else doesn''t make economic sense. regards Justin Forder --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To post to this group, send email to rubyonrails-core@googlegroups.com To unsubscribe from this group, send email to rubyonrails-core-unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/rubyonrails-core?hl=en -~----------~----~----~----~------~----~------~--~---
On 1/24/07, Justin Forder <justin@j-m-f.demon.co.uk> wrote:> > You are right to point out that variations in environment can cause a > test to fail for the creator of a ticket, but succeed when tested by a > Rails committer. > > But the situation you are concerned about seems to be when the ticket > creator hasn''t given a failing (for him/her) test. You are proposing > that the Rails committer who wants to close a ticket (i) interprets the > reported problem in terms of a test, (ii) writes the test, and (iii) > demonstrates that, in some ''standard'' environment, the test passes. > > In other words, you are proposing that Rails is presumed guilty until > proved innocent, and asking the committers to act as the lawyers for the > defendant. > > The number of Rails users is in the tens or hundreds of thousands, with > a wide variety of levels of skill and experience. You can see from > reading the mailing list how often people raise ''problems'' that turn out > to be the result of their own error or misunderstanding. > > The number of committers is small, and they are highly skilled. In the > circumstances, the committers *must* be able to close tickets without > spending a lot of time explaining why, and the burden must then fall on > the ticket creator to justify reopening the ticket. Anything else > doesn''t make economic sense.I am actually more concerned with people who are not committers closing tickets. One of the privileges of being a committer on a project is that you have gained enough trust to warrent the right to close tickets. Having said that, I still believe that *anyone* who wants to close a ticket due to lack of tests should at least mark it as untested and then close it. This is the common practice of the Rails committers and is even documented in the needy report. By marking it untested it appears in the needy report and can be followed up on by others when time permits. Marking it as worksforme or wontfix just seems wrong to me. Anyhow, we can discuss this until we''re blue in the face. At the end of the day what matters is how we behave. To that end *I* will continue to follow my own advice as to how to handle tickets for Rails, and I encourage others to do the same, if for no other reason than to contribute more test coverage to, and to inspire more confidence in the stability of, the best darn web framework out there. V/r Anthony Eden -- Cell: 808 782-5046 Current Location: Melbourne, FL --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To post to this group, send email to rubyonrails-core@googlegroups.com To unsubscribe from this group, send email to rubyonrails-core-unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/rubyonrails-core?hl=en -~----------~----~----~----~------~----~------~--~---
Anthony Eden wrote:> On 1/24/07, Justin Forder <justin@j-m-f.demon.co.uk> wrote: >> You are right to point out that variations in environment can cause a >> test to fail for the creator of a ticket, but succeed when tested by a >> Rails committer. >> >> But the situation you are concerned about seems to be when the ticket >> creator hasn''t given a failing (for him/her) test. You are proposing >> that the Rails committer who wants to close a ticket (i) interprets the >> reported problem in terms of a test, (ii) writes the test, and (iii) >> demonstrates that, in some ''standard'' environment, the test passes. >> >> In other words, you are proposing that Rails is presumed guilty until >> proved innocent, and asking the committers to act as the lawyers for the >> defendant. >> >> The number of Rails users is in the tens or hundreds of thousands, with >> a wide variety of levels of skill and experience. You can see from >> reading the mailing list how often people raise ''problems'' that turn out >> to be the result of their own error or misunderstanding. >> >> The number of committers is small, and they are highly skilled. In the >> circumstances, the committers *must* be able to close tickets without >> spending a lot of time explaining why, and the burden must then fall on >> the ticket creator to justify reopening the ticket. Anything else >> doesn''t make economic sense. > > I am actually more concerned with people who are not committers > closing tickets. One of the privileges of being a committer on a > project is that you have gained enough trust to warrent the right to > close tickets. Having said that, I still believe that *anyone* who > wants to close a ticket due to lack of tests should at least mark it > as untested and then close it. This is the common practice of the > Rails committers and is even documented in the needy report. By > marking it untested it appears in the needy report and can be followed > up on by others when time permits. Marking it as worksforme or wontfix > just seems wrong to me. > > Anyhow, we can discuss this until we''re blue in the face. At the end > of the day what matters is how we behave. To that end *I* will > continue to follow my own advice as to how to handle tickets for > Rails, and I encourage others to do the same, if for no other reason > than to contribute more test coverage to, and to inspire more > confidence in the stability of, the best darn web framework out there.Thanks, Anthony - that makes sense to me. regards Justin --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To post to this group, send email to rubyonrails-core@googlegroups.com To unsubscribe from this group, send email to rubyonrails-core-unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/rubyonrails-core?hl=en -~----------~----~----~----~------~----~------~--~---
> Anyhow, we can discuss this until we''re blue in the face. At the end > of the day what matters is how we behave. To that end *I* will > continue to follow my own advice as to how to handle tickets for > Rails, and I encourage others to do the same, if for no other reason > than to contribute more test coverage to, and to inspire more > confidence in the stability of, the best darn web framework out there.I think that in general, the onus is on the submitter to either provide a failing test case, or simple steps to reproduce. If a ticket gets closed prematurely, I figure there are a couple of potential results: 0) it was invalid, everyone wins 1) The submitter reopens it and provides more information on reproducibility, everyone wins 2) The submitter doesn''t care any more and the bug stays closed. everyone wins as we can go on to fix bugs which people care about. 3) The submitter doesn''t care any more, but someone submits a new bug. the bug''s either better documented, or the problem repeats. We don''t have a dearth of users, and we''re not penalising users who submit invalid bugs. So I figure, err on the side of closing, reopen with more information, not a bitchy comment. We''re all after the same thing here :) -- Cheers Koz --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To post to this group, send email to rubyonrails-core@googlegroups.com To unsubscribe from this group, send email to rubyonrails-core-unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/rubyonrails-core?hl=en -~----------~----~----~----~------~----~------~--~---
On Jan 24, 2007, at 6:09 PM, Michael Koziarski wrote:> >> Anyhow, we can discuss this until we''re blue in the face. At the end >> of the day what matters is how we behave. To that end *I* will >> continue to follow my own advice as to how to handle tickets for >> Rails, and I encourage others to do the same, if for no other reason >> than to contribute more test coverage to, and to inspire more >> confidence in the stability of, the best darn web framework out >> there. > > I think that in general, the onus is on the submitter to either > provide a failing test case, or simple steps to reproduce. If a > ticket gets closed prematurely, I figure there are a couple of > potential results: > > 0) it was invalid, everyone wins > 1) The submitter reopens it and provides more information on > reproducibility, everyone wins > 2) The submitter doesn''t care any more and the bug stays closed. > everyone wins as we can go on to fix bugs which people care about. > 3) The submitter doesn''t care any more, but someone submits a new bug. > the bug''s either better documented, or the problem repeats. > > We don''t have a dearth of users, and we''re not penalising users who > submit invalid bugs. So I figure, err on the side of closing, reopen > with more information, not a bitchy comment. We''re all after the same > thing here :)+1. If you''ve observed a bug and you really want it fixed, but can''t provide a patch (for whatever reason), a failing test case is the best way to "prove" that it is a bug. That''s not to say that every bug report must include a failing test case, but if you don''t provide it, you shouldn''t be angry if no one else can verify what you''ve reported. I believe this is a common philosophy in most OSS projects. - Jamis --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To post to this group, send email to rubyonrails-core@googlegroups.com To unsubscribe from this group, send email to rubyonrails-core-unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/rubyonrails-core?hl=en -~----------~----~----~----~------~----~------~--~---
On Jan 25, 2007, at 1:40 AM, Anthony Eden wrote:> I am actually more concerned with people who are not committers > closing tickets. One of the privileges of being a committer on a > project is that you have gained enough trust to warrent the right to > close tickets.+1 -- Julian ''Julik'' Tarkhanov please send all personal mail to me at julik.nl --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To post to this group, send email to rubyonrails-core@googlegroups.com To unsubscribe from this group, send email to rubyonrails-core-unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/rubyonrails-core?hl=en -~----------~----~----~----~------~----~------~--~---