R devel - Nov 2016 - Running package tests and not stop on first fail

>>>>> Dirk Eddelbuettel <edd at debian.org>
>>>>>     on Fri, 4 Nov 2016 10:36:52 -0500 writes:
    > On 4 November 2016 at 16:24, Martin Maechler wrote: | My
    > proposed name '--no-stop-on-error' was a quick shot; if |
    > somebody has a more concise or better "English style"
    > wording | (which is somewhat compatible with all the other
    > options you see | from 'R CMD check --help'), | please
    > speak up.

    > Why not keep it simple?  The similar feature this most
    > resembles is 'make -k' and its help page has

    >        -k, --keep-going
       
    >                    Continue as much as possible after an
    > error.  While the target that failed, and those that
    > depend on it, cannot be remade, the other dependencies of
    > these targets can be processed all the same.

Yes, that would be quite a bit simpler and nice in my view.
One may think it to be too vague,
notably from Brian Pedersen's mentioning that the examples are
already continued in any case if they lead to an error.

Other opinions?

On Friday, 4 November 2016, Martin Maechler <maechler at
stat.math.ethz.ch>
wrote:
> >>>>> Dirk Eddelbuettel <edd at debian.org
<javascript:;>>
> >>>>>     on Fri, 4 Nov 2016 10:36:52 -0500 writes:
>
>     > On 4 November 2016 at 16:24, Martin Maechler wrote: | My
>     > proposed name '--no-stop-on-error' was a quick shot; if |
>     > somebody has a more concise or better "English style"
>     > wording | (which is somewhat compatible with all the other
>     > options you see | from 'R CMD check --help'), | please
>     > speak up.
>
>     > Why not keep it simple?  The similar feature this most
>     > resembles is 'make -k' and its help page has
>
>     >        -k, --keep-going
>
>     >                    Continue as much as possible after an
>     > error.  While the target that failed, and those that
>     > depend on it, cannot be remade, the other dependencies of
>     > these targets can be processed all the same.
>
> Yes, that would be quite a bit simpler and nice in my view.
> One may think it to be too vague,

Mmn, I would agree on vagueness (and it breaks the pattern set by other
flags of human-readability). Deep familiarity with make is probably not
something we should ask of everyone who needs to test a package, too.

I quite like stop-on-error=true (exactly the same as the previous
suggestion but shaves off some characters by inverting the Boolean)

notably from Brian Pedersen's mentioning that the examples
are
> already continued in any case if they lead to an error.
>
> Other opinions?
>
> ______________________________________________
> R-devel at r-project.org <javascript:;> mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
	[[alternative HTML version deleted]]

>>>>> Oliver Keyes <ironholds at gmail.com>
>>>>>     on Fri, 4 Nov 2016 12:42:54 -0400 writes:
    > On Friday, 4 November 2016, Martin Maechler
    > <maechler at stat.math.ethz.ch> wrote:

    >> >>>>> Dirk Eddelbuettel <edd at debian.org
<javascript:;>>
    >> >>>>> on Fri, 4 Nov 2016 10:36:52 -0500 writes:
    >> 
    >> > On 4 November 2016 at 16:24, Martin Maechler wrote: |
    >> My > proposed name '--no-stop-on-error' was a quick
shot;
    >> if | > somebody has a more concise or better "English
    >> style" > wording | (which is somewhat compatible with all
    >> the other > options you see | from 'R CMD check
--help'),
    >> | please > speak up.
    >> 
    >> > Why not keep it simple?  The similar feature this most
    >> > resembles is 'make -k' and its help page has
    >> 
    >> > -k, --keep-going
    >> 
    >> > Continue as much as possible after an > error.  While
    >> the target that failed, and those that > depend on it,
    >> cannot be remade, the other dependencies of > these
    >> targets can be processed all the same.
    >> 
    >> Yes, that would be quite a bit simpler and nice in my
    >> view.  One may think it to be too vague,

    > Mmn, I would agree on vagueness (and it breaks the pattern
    > set by other flags of human-readability). Deep familiarity
    > with make is probably not something we should ask of
    > everyone who needs to test a package, too.

    > I quite like stop-on-error=true (exactly the same as the
    > previous suggestion but shaves off some characters by
    > inverting the Boolean)

Thank you, Brian, Dirk and Oliver for these (and some offline)
thoughts and suggestions!

My current summary:

1) I really don't want a  --<option-key>=value
   but rather stay with logical/binary variables that "express
   themselves"... in the same way I strongly prefer

       if (A_is_special)       ....
   to
       if (A_special == TRUE)  ....

   for a logical variable A_* .   Yes, this is mostly a matter
   of taste,.. but related to how R style itself "works"

2) Brian mentioned that this is only about ./tests/ tests which
   are continued, not about the Examples which are treated separately.
   That's why we had contemplated additionally using 'tests'
(because that's
   the directory name used for unit/regression/.. tests) in the option
   name.

Even though Brian is correct, ideally we *would* want to also influence the
examples' running to *not* stop on a first error..   However that would
need more work, reorganizing how the examples are run and that may not be
worth the pain.   However it should be considered a goal in the long run.

After all that, I tend to remain with the original proposed name. It is at
least easy to pronounce and spell correctly...

Martin

Sorry for late reply. I like the stop-on-error.
Thanks for merging.
Glad to be R contributor!

On 4 November 2016 at 09:42, Oliver Keyes <ironholds at gmail.com>
wrote:
> On Friday, 4 November 2016, Martin Maechler <maechler at
stat.math.ethz.ch>
> wrote:
>>
>> >>>>> Dirk Eddelbuettel <edd at debian.org>
>> >>>>>     on Fri, 4 Nov 2016 10:36:52 -0500 writes:
>>
>>     > On 4 November 2016 at 16:24, Martin Maechler wrote: | My
>>     > proposed name '--no-stop-on-error' was a quick shot;
if |
>>     > somebody has a more concise or better "English
style"
>>     > wording | (which is somewhat compatible with all the other
>>     > options you see | from 'R CMD check --help'), | please
>>     > speak up.
>>
>>     > Why not keep it simple?  The similar feature this most
>>     > resembles is 'make -k' and its help page has
>>
>>     >        -k, --keep-going
>>
>>     >                    Continue as much as possible after an
>>     > error.  While the target that failed, and those that
>>     > depend on it, cannot be remade, the other dependencies of
>>     > these targets can be processed all the same.
>>
>> Yes, that would be quite a bit simpler and nice in my view.
>> One may think it to be too vague,
>
>
> Mmn, I would agree on vagueness (and it breaks the pattern set by other
> flags of human-readability). Deep familiarity with make is probably not
> something we should ask of everyone who needs to test a package, too.
>
> I quite like stop-on-error=true (exactly the same as the previous
suggestion
> but shaves off some characters by inverting the Boolean)
>
>> notably from Brian Pedersen's mentioning that the examples are
>> already continued in any case if they lead to an error.
>>
>> Other opinions?
>>
>> ______________________________________________
>> R-devel at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel