R devel - Aug 2021 - [External] Re: Update on rtools4 and ucrt support

On Mon, 23 Aug 2021, Duncan Murdoch wrote:
> On 23/08/2021 8:15 a.m., jan Vitek via R-devel wrote:
>> Hi Jeroen,
>> 
>> I mostly lurk on this list, but I was struck by your combative tone.
>> 
>> To pick on two random bits:
>> 
>>> ? a 6gb tarball with manually built things on his personal machine?
>> 
>>> ? a black-box system that is so opaque and complex that only one
person
>>> knows how it works, and would make it much more difficult for
>>> students, universities, and other organisations to build R packages
>>> and libraries on Windows?
>> 
>> 
>> Tomas? tool chain isn't a blackbox, it has copious documentation
(see [1])
>> and builds on any machine thanks to the provided docker container.
>> 
>> This is not to criticise your work which has its unique strengths, but
to
>> state the obvious: these strengths are best discussed without passion
>> based on factually accurate descriptions.
>
> I agree with Jan.  I'm not sure a discussion in this forum would be
fruitful,
> but I really wish Jeroen and Tomas would get together, aiming to merge
their
> toolchains, keeping the best aspects of both.
>
> I haven't been involved in the development of either one, but have been
a
> "victim" of the two chain rivalry, because the rgl package is not
easy to
> build.  I get instructions from each of them on how to do the build, and 
> those instructions for one toolchain generally break the build on the other
> one.  While it is probably possible to detect the toolchain and have the 
> build adapt to whichever one is in use, it would be a lot easier for me
(and
> I imagine every other maintainer of a package using external libs) if I
just
> had to follow one set of instructions.
>
> Duncan Murdoch
Here are just a few comments from my perspective (I am an R-core
member, but am not part of the CRAN team and do only very limited work
on Windows). Other R-core members may have different perspectives and
insights.

One bit of background: dealing with encoding issues on Windows has
been taking an unsustainable amount of R-core resources for some time
now. Tomas Kalibera has been taking the lead on trying to address
these issues in the existing framework, but this means he has not had
the time to make any of the many other valuable and important
contributions he could make. The only viable way forward is to move to
a Windows tool chain that supports UTF-8 as the C library current
encoding via the Windows UCRT framework.

Tomas Kalibera has, on behalf of all of R core and in
coordination with CRAN, been looking for a way forward for some
time and has reported on the progress in several blog posts at
https://developer.r-project.org/Blog/public/. This has lead to
the development of the MXE-based UCRT tool chain, which is now
well tested and ready for deployment.  Checks using the UCRT tool
chain have been part of the CRAN check process for a while. I
believe CRAN plans to switch R-devel checks and builds to the
UCRT tool chain during the upcoming CRAN downtime. I expect there
will be some communication from CRAN on this soon, including on
any issues in supporting binaries for both R-devel and R-patched.

In putting together something as large as a tool chain there will
always be many choices, each with advantages and disadvantages.  Some
things may be advantages in some settings and not others. Taking just
one case in point: Cross compilation. This is likely to be a better
approach for CRAN in the future and is supported by the MXE framework
on which the new tool chain is based.

The much more recent changes in rtools4 to support UCRT are at this
point not yet as well tested as the new tool chain. Once these changes
to rtools4 mature, and if binary compatibility can be assured, then
having a second tool chain may be useful in some cases.  But if there
are incompatibilities then it will be up to rtools4 to keep up with
the tool chain used by CRAN. On the other, contributing to improving
the MXE-based tool chain may be a better investment of time.

Best,

luke
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
-- 
Luke Tierney
Ralph E. Wareham Professor of Mathematical Sciences
University of Iowa                  Phone:             319-335-3386
Department of Statistics and        Fax:               319-335-3017
    Actuarial Science
241 Schaeffer Hall                  email:   luke-tierney at uiowa.edu
Iowa City, IA 52242                 WWW:  http://www.stat.uiowa.edu

On Mon, Aug 23, 2021 at 11:09 PM <luke-tierney at uiowa.edu> wrote:
> On Mon, 23 Aug 2021, Duncan Murdoch wrote:
>
> > On 23/08/2021 8:15 a.m., jan Vitek via R-devel wrote:
> >> Hi Jeroen,
> >>
> >> I mostly lurk on this list, but I was struck by your combative
tone.
> >>
> >> To pick on two random bits:
> >>
> >>> ? a 6gb tarball with manually built things on his personal
machine?
> >>
> >>> ? a black-box system that is so opaque and complex that only
one person
> >>> knows how it works, and would make it much more difficult for
> >>> students, universities, and other organisations to build R
packages
> >>> and libraries on Windows?
> >>
> >>
> >> Tomas? tool chain isn't a blackbox, it has copious
documentation (see
> [1])
> >> and builds on any machine thanks to the provided docker container.
> >>
> >> This is not to criticise your work which has its unique strengths,
but
> to
> >> state the obvious: these strengths are best discussed without
passion
> >> based on factually accurate descriptions.
> >
> > I agree with Jan.  I'm not sure a discussion in this forum would
be
> fruitful,
> > but I really wish Jeroen and Tomas would get together, aiming to merge
> their
> > toolchains, keeping the best aspects of both.
> >
> > I haven't been involved in the development of either one, but have
been
> a
> > "victim" of the two chain rivalry, because the rgl package
is not easy
> to
> > build.  I get instructions from each of them on how to do the build,
and
> > those instructions for one toolchain generally break the build on the
> other
> > one.  While it is probably possible to detect the toolchain and have
the
> > build adapt to whichever one is in use, it would be a lot easier for
me
> (and
> > I imagine every other maintainer of a package using external libs) if
I
> just
> > had to follow one set of instructions.
> >
> > Duncan Murdoch
>
> Here are just a few comments from my perspective (I am an R-core
> member, but am not part of the CRAN team and do only very limited work
> on Windows). Other R-core members may have different perspectives and
> insights.
>
> One bit of background: dealing with encoding issues on Windows has
> been taking an unsustainable amount of R-core resources for some time
> now. Tomas Kalibera has been taking the lead on trying to address
> these issues in the existing framework, but this means he has not had
> the time to make any of the many other valuable and important
> contributions he could make. The only viable way forward is to move to
> a Windows tool chain that supports UTF-8 as the C library current
> encoding via the Windows UCRT framework.
>
> Tomas Kalibera has, on behalf of all of R core and in
> coordination with CRAN, been looking for a way forward for some
> time and has reported on the progress in several blog posts at
> https://developer.r-project.org/Blog/public/. This has lead to
> the development of the MXE-based UCRT tool chain, which is now
> well tested and ready for deployment.  Checks using the UCRT tool
> chain have been part of the CRAN check process for a while. I
> believe CRAN plans to switch R-devel checks and builds to the
> UCRT tool chain during the upcoming CRAN downtime. I expect there
> will be some communication from CRAN on this soon, including on
> any issues in supporting binaries for both R-devel and R-patched.
>
> In putting together something as large as a tool chain there will
> always be many choices, each with advantages and disadvantages.  Some
> things may be advantages in some settings and not others. Taking just
> one case in point: Cross compilation. This is likely to be a better
> approach for CRAN in the future and is supported by the MXE framework
> on which the new tool chain is based.
>
> The much more recent changes in rtools4 to support UCRT are at this
> point not yet as well tested as the new tool chain. Once these changes
> to rtools4 mature, and if binary compatibility can be assured, then
> having a second tool chain may be useful in some cases.  But if there
> are incompatibilities then it will be up to rtools4 to keep up with
> the tool chain used by CRAN. On the other, contributing to improving
> the MXE-based tool chain may be a better investment of time.
>
> Best,
>
> luke
>
> >
> > ______________________________________________
> > R-devel at r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-devel
> >
>
> --
> Luke Tierney
> Ralph E. Wareham Professor of Mathematical Sciences
> University of Iowa                  Phone:             319-335-3386
> Department of Statistics and        Fax:               319-335-3017
>     Actuarial Science
> 241 Schaeffer Hall                  email:   luke-tierney at uiowa.edu
> Iowa City, IA 52242                 WWW:  http://www.stat.uiowa.edu
>
Thank you, Dr. Tierney. However, I am concerned about the not-insignificant
number of us who for various reasons can only do our development on
Windows. Rtools has been the official tool chain with which to build
windows for the at least 20 years I have been using R (yes, a babe in the
woods compared to most, but not a complete neophyte). Duncan and Jereoen
have each done yeoman?s jobs in ensuring that R can be built from source on
Windows and that packages can be developed for all OSS on Windows?even
Solaris SPARC.

I am much less aware of Thomas?s work, and I?ll gladly take the blame for
it, but I haven?t seen an accessible tool chain built by him which would
allow me, the Windows developer, to build R and all packages from source on
a native Windows box. Have I just missed it? If not, is that planned? If
R-core switches the official Windows toolchain, where does that leave us?

Thank you,

Avi

-- 
Sent from Gmail Mobile

	[[alternative HTML version deleted]]