R devel - Dec 2017 - R CMD check warning about compiler warning flags

>>>>> Duncan Murdoch <murdoch.duncan at gmail.com>
>>>>>     on Thu, 21 Dec 2017 14:23:13 -0500 writes:
    > On 21/12/2017 1:02 PM, Winston Chang wrote:
    >>>> On recent builds of R-devel, R CMD check gives a
    >>>> WARNING when some compiler warning flags are detected,
    >>>> such as -Werror, because they are non-portable. This
    >>>> appears to have been added in this commit:
    >>>> https://github.com/wch/r-source/commit/2e80059
    >>> 
    >>> That is not the canonical R sources.
    >> 
    >> Yes, that is obvious. The main page for that repository
    >> says it is a mirror of the R sources, right at the top. I
    >> know that because I put the message there, and because I
    >> see it every time I visit the repository. If you have a
    >> good way of pointing people to the changes made in a
    >> commit with the canonical R sources, please let us
    >> know. I and many others would be happy to use it.

    > The usual way is just to refer to the revision number,
    > i.e. "This appears to have been added in rev 73909".

    > People who don't have the sources checked out can see the
    > diff on your Github mirror using

    > https://github.com/wch/r-source/search?q="trunk at
73909"&type=Commits

    > and following the listed search hit. (Thanks to Thierry
    > Onkelinx for helping me with this.) This only works for
    > commits to the trunk.  I guessed that something like

    > https://github.com/wch/r-source/search?q="R-3-4-branch at
73937"&type=Commits

    > would work if the commit was to the 3.4 branch, but
    > apparently not.  I don't know how to find those commits.
    > Presumably there's a way, but I don't know it.

    > Another possibility is that someone could set up (or
    > already has?) one of the web viewers (WebSVN, etc.) for
    > the real repository.  That would be better for those of us
    > who are SVN users, but probably harder for Git users.

    > Duncan Murdoch

As you know I had setup (the first few versions of) the svn at
  https://svn.r-project.org/

at the time, I wanted to keep that machine protected as much as
possible and had decided not to install any other apache
modules and svn - niceties just such that the server would run
minimal services and hence would be minimally vulnerable.

The times have changed though and I will look into adding WebSVN
to svn.r-project.org  as one of the  first things in 2018.

Martin Maechler




    >> 
    >>> And your description seems wrong: there is now an
    >>> _optional_ check controlled by an environment variable,
    >>> primarily for CRAN checks.
    >> 
    >> The check is "optional", but not for packages submitted
    >> to CRAN.
    >> 
    >> 
    >>>> I'm working on a package where these compiler warning
    >>>> flags are present in a Makefile generated by a
    >>>> configure script -- that is, the configure script
    >>>> detects whether the compiler supports these flags, and
    >>>> if so, puts them in the Makefile. (The configure script
    >>>> is for a third-party C library which is in a
    >>>> subdirectory of src/.)
    >>>> 
    >>>> Because the flags are added only if the system supports
    >>>> them, there shouldn't be any worries about portability
    >>>> in practice.
    >>> 
    >>> 
    >>> Please read the explanation in the manual: there are
    >>> serious concerns about such flags which have bitten CRAN
    >>> users several times.
    >>> 
    >>> To take your example, you cannot know what -Werror does
    >>> on all compilers (past, present or future) where it is
    >>> supported (and -W flags do do different things on
    >>> different compilers).  On current gcc it does
    >>> 
    >>> -Werror Make all warnings into errors.
    >>> 
    >>> and so its effect depends on what other flags are used
    >>> (people typically use -Wall, and most new versions of
    >>> both gcc and clang add more warnings to -Wall -- I read
    >>> this week exactly such a discussion about the
    >>> interaction of -Werror with
    >>> -Wtautological-constant-compare as part of -Wall in
    >>> clang trunk).
    >>> 
    >>>> Is there a way to get R CMD check to not raise warnings
    >>>> in cases like this? I know I could modify the C
    >>>> library's configure.ac (which is used to generate the
    >>>> configure script) but I'd prefer to leave the
library's
    >>>> code untouched if possible.
    >>> 
    >>> You don't need to (and most likely should not) use the
    >>> C[XX]FLAGS it generates ... just use the flags which R
    >>> passes to the package to use.
    >> 
    >> It turns out that there isn't even a risk of these
    >> compiler flags being used -- I learned from of my
    >> colleagues that the troublesome compiler flags, like
    >> -Werror, never actually appear in the Makefile.  The
    >> configure script prints out those compiler flags out when
    >> it checks for them, but in the end it creates a Makefile
    >> with the CFLAGS inherited from R. So there's no chance
    >> that the library would be compiled using those flags
    >> (unless R passed them along).
    >> 
    >> His suggested workaround is to silence the output of the
    >> configure script. That also hides some useful
    >> information, but it does work for this issue.
    >> 
    >> -Winston
    >> 
    >> ______________________________________________
    >> R-devel at r-project.org mailing list
    >> https://stat.ethz.ch/mailman/listinfo/r-devel
    >> 

    > ______________________________________________
    > R-devel at r-project.org mailing list
    > https://stat.ethz.ch/mailman/listinfo/r-devel

Hi,

See inline below.

> On Dec 22, 2017, at 9:12 AM, Martin Maechler <maechler at
stat.math.ethz.ch> wrote:
> 
>>>>>> Duncan Murdoch <murdoch.duncan at gmail.com>
>>>>>>    on Thu, 21 Dec 2017 14:23:13 -0500 writes:
> 
>> On 21/12/2017 1:02 PM, Winston Chang wrote:
>>>>> On recent builds of R-devel, R CMD check gives a
>>>>> WARNING when some compiler warning flags are detected,
>>>>> such as -Werror, because they are non-portable. This
>>>>> appears to have been added in this commit:
>>>>> https://github.com/wch/r-source/commit/2e80059
>>>> 
>>>> That is not the canonical R sources.
>>> 
>>> Yes, that is obvious. The main page for that repository
>>> says it is a mirror of the R sources, right at the top. I
>>> know that because I put the message there, and because I
>>> see it every time I visit the repository. If you have a
>>> good way of pointing people to the changes made in a
>>> commit with the canonical R sources, please let us
>>> know. I and many others would be happy to use it.
> 
>> The usual way is just to refer to the revision number,
>> i.e. "This appears to have been added in rev 73909".
> 
>> People who don't have the sources checked out can see the
>> diff on your Github mirror using
> 
>> https://github.com/wch/r-source/search?q="trunk at
73909"&type=Commits
> 
>> and following the listed search hit. (Thanks to Thierry
>> Onkelinx for helping me with this.) This only works for
>> commits to the trunk.  I guessed that something like
> 
>> https://github.com/wch/r-source/search?q="R-3-4-branch at
73937"&type=Commits
> 
>> would work if the commit was to the 3.4 branch, but
>> apparently not.  I don't know how to find those commits.
>> Presumably there's a way, but I don't know it.
> 
>> Another possibility is that someone could set up (or
>> already has?) one of the web viewers (WebSVN, etc.) for
>> the real repository.  That would be better for those of us
>> who are SVN users, but probably harder for Git users.
> 
>> Duncan Murdoch
> 
> As you know I had setup (the first few versions of) the svn at
>  https://svn.r-project.org/
> 
> at the time, I wanted to keep that machine protected as much as
> possible and had decided not to install any other apache
> modules and svn - niceties just such that the server would run
> minimal services and hence would be minimally vulnerable.
> 
> The times have changed though and I will look into adding WebSVN
> to svn.r-project.org  as one of the  first things in 2018.
> 
> Martin Maechler

Martin,

Just to play a bit of a devil's advocate here, WebSVN has not been
updated/maintained since June of 2011, so is a number of years old at this
point. That should raise some reasonable concerns over bugs, security issues and
related matters.

To put that in perspective, the last update to WebSVN was around the time that R
2.13.1 was released.

That general pattern, of SVN clients not being actively maintained, seems to be
consistent across a number of the web based (and even browser plugin based) SVN
clients, which may in turn, be an indication of the general shift to Git in
recent years, much as the shift from CVS to SVN occurred years ago.

In researching other SVN clients, the few that still seem to be actively
maintained are typically dedicated desktop clients/GUIs, that in some cases are
closed source and/or are OS specific.

One of the few web based SVN clients that still seems to be actively maintained
is Trac (https://trac.edgewall.org), however it is under a modified BSD license,
which may raise some issues and from what I can tell, may be more complicated in
terms of set up, since it also supports bug tracking, wiki and other
functionality. As with any such application, there would be ongoing maintenance
issues as well.

I am not advocating any particular solution. I just want to point out potential
issues with WebSVN and raise for discussion, what options may make sense to
consider, based upon how many folks might actually use such "live" web
based functionality versus remote SVN access via a desktop client, which is
certainly an option, since they won't have write access anyway.

Also, a number of the Git desktop clients also support Git SVN
(https://git-scm.com/docs/git-svn), for compatibility, which offers Git users an
alternative to using a dedicated desktop SVN client.

Regards,

Marc

> 
>>> 
>>>> And your description seems wrong: there is now an
>>>> _optional_ check controlled by an environment variable,
>>>> primarily for CRAN checks.
>>> 
>>> The check is "optional", but not for packages submitted
>>> to CRAN.
>>> 
>>> 
>>>>> I'm working on a package where these compiler warning
>>>>> flags are present in a Makefile generated by a
>>>>> configure script -- that is, the configure script
>>>>> detects whether the compiler supports these flags, and
>>>>> if so, puts them in the Makefile. (The configure script
>>>>> is for a third-party C library which is in a
>>>>> subdirectory of src/.)
>>>>> 
>>>>> Because the flags are added only if the system supports
>>>>> them, there shouldn't be any worries about portability
>>>>> in practice.
>>>> 
>>>> 
>>>> Please read the explanation in the manual: there are
>>>> serious concerns about such flags which have bitten CRAN
>>>> users several times.
>>>> 
>>>> To take your example, you cannot know what -Werror does
>>>> on all compilers (past, present or future) where it is
>>>> supported (and -W flags do do different things on
>>>> different compilers).  On current gcc it does
>>>> 
>>>> -Werror Make all warnings into errors.
>>>> 
>>>> and so its effect depends on what other flags are used
>>>> (people typically use -Wall, and most new versions of
>>>> both gcc and clang add more warnings to -Wall -- I read
>>>> this week exactly such a discussion about the
>>>> interaction of -Werror with
>>>> -Wtautological-constant-compare as part of -Wall in
>>>> clang trunk).
>>>> 
>>>>> Is there a way to get R CMD check to not raise warnings
>>>>> in cases like this? I know I could modify the C
>>>>> library's configure.ac (which is used to generate the
>>>>> configure script) but I'd prefer to leave the
library's
>>>>> code untouched if possible.
>>>> 
>>>> You don't need to (and most likely should not) use the
>>>> C[XX]FLAGS it generates ... just use the flags which R
>>>> passes to the package to use.
>>> 
>>> It turns out that there isn't even a risk of these
>>> compiler flags being used -- I learned from of my
>>> colleagues that the troublesome compiler flags, like
>>> -Werror, never actually appear in the Makefile.  The
>>> configure script prints out those compiler flags out when
>>> it checks for them, but in the end it creates a Makefile
>>> with the CFLAGS inherited from R. So there's no chance
>>> that the library would be compiled using those flags
>>> (unless R passed them along).
>>> 
>>> His suggested workaround is to silence the output of the
>>> configure script. That also hides some useful
>>> information, but it does work for this issue.
>>> 
>>> -Winston

On 22/12/2017 10:46 AM, Marc Schwartz wrote:
> Hi,
> 
> See inline below.
> 
> 
>> On Dec 22, 2017, at 9:12 AM, Martin Maechler <maechler at
stat.math.ethz.ch> wrote:
>>
>>>>>>> Duncan Murdoch <murdoch.duncan at gmail.com>
>>>>>>>     on Thu, 21 Dec 2017 14:23:13 -0500 writes:
>>
>>> On 21/12/2017 1:02 PM, Winston Chang wrote:
>>>>>> On recent builds of R-devel, R CMD check gives a
>>>>>> WARNING when some compiler warning flags are detected,
>>>>>> such as -Werror, because they are non-portable. This
>>>>>> appears to have been added in this commit:
>>>>>> https://github.com/wch/r-source/commit/2e80059
>>>>>
>>>>> That is not the canonical R sources.
>>>>
>>>> Yes, that is obvious. The main page for that repository
>>>> says it is a mirror of the R sources, right at the top. I
>>>> know that because I put the message there, and because I
>>>> see it every time I visit the repository. If you have a
>>>> good way of pointing people to the changes made in a
>>>> commit with the canonical R sources, please let us
>>>> know. I and many others would be happy to use it.
>>
>>> The usual way is just to refer to the revision number,
>>> i.e. "This appears to have been added in rev 73909".
>>
>>> People who don't have the sources checked out can see the
>>> diff on your Github mirror using
>>
>>> https://github.com/wch/r-source/search?q="trunk at
73909"&type=Commits
>>
>>> and following the listed search hit. (Thanks to Thierry
>>> Onkelinx for helping me with this.) This only works for
>>> commits to the trunk.  I guessed that something like
>>
>>> https://github.com/wch/r-source/search?q="R-3-4-branch at
73937"&type=Commits
>>
>>> would work if the commit was to the 3.4 branch, but
>>> apparently not.  I don't know how to find those commits.
>>> Presumably there's a way, but I don't know it.
>>
>>> Another possibility is that someone could set up (or
>>> already has?) one of the web viewers (WebSVN, etc.) for
>>> the real repository.  That would be better for those of us
>>> who are SVN users, but probably harder for Git users.
>>
>>> Duncan Murdoch
>>
>> As you know I had setup (the first few versions of) the svn at
>>   https://svn.r-project.org/
>>
>> at the time, I wanted to keep that machine protected as much as
>> possible and had decided not to install any other apache
>> modules and svn - niceties just such that the server would run
>> minimal services and hence would be minimally vulnerable.
>>
>> The times have changed though and I will look into adding WebSVN
>> to svn.r-project.org  as one of the  first things in 2018.
>>
>> Martin Maechler
> 
> 
> Martin,
> 
> Just to play a bit of a devil's advocate here, WebSVN has not been
updated/maintained since June of 2011, so is a number of years old at this
point. That should raise some reasonable concerns over bugs, security issues and
related matters.
> 
> To put that in perspective, the last update to WebSVN was around the time
that R 2.13.1 was released.
> 
> That general pattern, of SVN clients not being actively maintained, seems
to be consistent across a number of the web based (and even browser plugin
based) SVN clients, which may in turn, be an indication of the general shift to
Git in recent years, much as the shift from CVS to SVN occurred years ago.
> 
> In researching other SVN clients, the few that still seem to be actively
maintained are typically dedicated desktop clients/GUIs, that in some cases are
closed source and/or are OS specific.
> 
> One of the few web based SVN clients that still seems to be actively
maintained is Trac (https://trac.edgewall.org), however it is under a modified
BSD license, which may raise some issues and from what I can tell, may be more
complicated in terms of set up, since it also supports bug tracking, wiki and
other functionality. As with any such application, there would be ongoing
maintenance issues as well.
There's also ViewVC which still seems to be active.
> I am not advocating any particular solution. I just want to point out
potential issues with WebSVN and raise for discussion, what options may make
sense to consider, based upon how many folks might actually use such
"live" web based functionality versus remote SVN access via a desktop
client, which is certainly an option, since they won't have write access
anyway.
As a general principle in open source projects the work should be done 
by the people who want to use the product.  Winston wanted git access to 
the R sources, and he put it together: bravo!

It's unfortunate that git doesn't really support svn completely, because
it is superior in many ways.  However, it isn't uniformly better than 
svn. Its very weak support for sequential labels means that switching to 
it would be a lot of work for R Core, and that group doesn't have a lot 
of spare time to waste.

The one negative aspect of Winston's effort is caused by this weakness. 
If you tell me that something happened in revision 73909, I know it was 
recent.  If you tell me that something appeared in commit 2e80059, it 
wastes my time looking up that commit and putting it in context.

Duncan Murdoch
> 
> Also, a number of the Git desktop clients also support Git SVN
(https://git-scm.com/docs/git-svn), for compatibility, which offers Git users an
alternative to using a dedicated desktop SVN client.
> 
> Regards,
> 
> Marc
> 
> 
>>
>>>>
>>>>> And your description seems wrong: there is now an
>>>>> _optional_ check controlled by an environment variable,
>>>>> primarily for CRAN checks.
>>>>
>>>> The check is "optional", but not for packages
submitted
>>>> to CRAN.
>>>>
>>>>
>>>>>> I'm working on a package where these compiler
warning
>>>>>> flags are present in a Makefile generated by a
>>>>>> configure script -- that is, the configure script
>>>>>> detects whether the compiler supports these flags, and
>>>>>> if so, puts them in the Makefile. (The configure script
>>>>>> is for a third-party C library which is in a
>>>>>> subdirectory of src/.)
>>>>>>
>>>>>> Because the flags are added only if the system supports
>>>>>> them, there shouldn't be any worries about
portability
>>>>>> in practice.
>>>>>
>>>>>
>>>>> Please read the explanation in the manual: there are
>>>>> serious concerns about such flags which have bitten CRAN
>>>>> users several times.
>>>>>
>>>>> To take your example, you cannot know what -Werror does
>>>>> on all compilers (past, present or future) where it is
>>>>> supported (and -W flags do do different things on
>>>>> different compilers).  On current gcc it does
>>>>>
>>>>> -Werror Make all warnings into errors.
>>>>>
>>>>> and so its effect depends on what other flags are used
>>>>> (people typically use -Wall, and most new versions of
>>>>> both gcc and clang add more warnings to -Wall -- I read
>>>>> this week exactly such a discussion about the
>>>>> interaction of -Werror with
>>>>> -Wtautological-constant-compare as part of -Wall in
>>>>> clang trunk).
>>>>>
>>>>>> Is there a way to get R CMD check to not raise warnings
>>>>>> in cases like this? I know I could modify the C
>>>>>> library's configure.ac (which is used to generate
the
>>>>>> configure script) but I'd prefer to leave the
library's
>>>>>> code untouched if possible.
>>>>>
>>>>> You don't need to (and most likely should not) use the
>>>>> C[XX]FLAGS it generates ... just use the flags which R
>>>>> passes to the package to use.
>>>>
>>>> It turns out that there isn't even a risk of these
>>>> compiler flags being used -- I learned from of my
>>>> colleagues that the troublesome compiler flags, like
>>>> -Werror, never actually appear in the Makefile.  The
>>>> configure script prints out those compiler flags out when
>>>> it checks for them, but in the end it creates a Makefile
>>>> with the CFLAGS inherited from R. So there's no chance
>>>> that the library would be compiled using those flags
>>>> (unless R passed them along).
>>>>
>>>> His suggested workaround is to silence the output of the
>>>> configure script. That also hides some useful
>>>> information, but it does work for this issue.
>>>>
>>>> -Winston
>