Tomas Kalibera
2021-Aug-23 22:22 UTC
[Rd] [External] Re: Update on rtools4 and ucrt support
On 8/24/21 12:03 AM, Avraham Adler wrote:> On Tue, Aug 24, 2021 at 12:47 AM Simon Urbanek <simon.urbanek at r-project.org> > wrote: > >> Avi, >> >> please see the announcement: >> >> >> https://developer.r-project.org/Blog/public/2021/03/12/windows/utf-8-toolchain-and-cran-package-checks/index.html >> >> the documentation is in >> >> >> https://svn.r-project.org/R-dev-web/trunk/WindowsBuilds/winutf8/ucrt3/howto.html >> >> Cheers, >> Simon >> >> >> >> >> On Aug 24, 2021, at 8:34 AM, Avraham Adler <avraham.adler at gmail.com> >> wrote: >> >> On Mon, Aug 23, 2021 at 11:09 PM <luke-tierney at uiowa.edu> wrote: >> >> On Mon, 23 Aug 2021, Duncan Murdoch wrote: >> >> On 23/08/2021 8:15 a.m., jan Vitek via R-devel wrote: >> >> Hi Jeroen, >> >> I mostly lurk on this list, but I was struck by your combative tone. >> >> To pick on two random bits: >> >> ? a 6gb tarball with manually built things on his personal machine? >> >> >> ? a black-box system that is so opaque and complex that only one person >> knows how it works, and would make it much more difficult for >> students, universities, and other organisations to build R packages >> and libraries on Windows? >> >> >> >> Tomas? tool chain isn't a blackbox, it has copious documentation (see >> >> [1]) >> >> and builds on any machine thanks to the provided docker container. >> >> This is not to criticise your work which has its unique strengths, but >> >> to >> >> state the obvious: these strengths are best discussed without passion >> based on factually accurate descriptions. >> >> >> I agree with Jan. I'm not sure a discussion in this forum would be >> >> fruitful, >> >> but I really wish Jeroen and Tomas would get together, aiming to merge >> >> their >> >> toolchains, keeping the best aspects of both. >> >> I haven't been involved in the development of either one, but have been >> >> a >> >> "victim" of the two chain rivalry, because the rgl package is not easy >> >> to >> >> build. I get instructions from each of them on how to do the build, and >> those instructions for one toolchain generally break the build on the >> >> other >> >> one. While it is probably possible to detect the toolchain and have the >> build adapt to whichever one is in use, it would be a lot easier for me >> >> (and >> >> I imagine every other maintainer of a package using external libs) if I >> >> just >> >> had to follow one set of instructions. >> >> Duncan Murdoch >> >> >> Here are just a few comments from my perspective (I am an R-core >> member, but am not part of the CRAN team and do only very limited work >> on Windows). Other R-core members may have different perspectives and >> insights. >> >> One bit of background: dealing with encoding issues on Windows has >> been taking an unsustainable amount of R-core resources for some time >> now. Tomas Kalibera has been taking the lead on trying to address >> these issues in the existing framework, but this means he has not had >> the time to make any of the many other valuable and important >> contributions he could make. The only viable way forward is to move to >> a Windows tool chain that supports UTF-8 as the C library current >> encoding via the Windows UCRT framework. >> >> Tomas Kalibera has, on behalf of all of R core and in >> coordination with CRAN, been looking for a way forward for some >> time and has reported on the progress in several blog posts at >> https://developer.r-project.org/Blog/public/. This has lead to >> the development of the MXE-based UCRT tool chain, which is now >> well tested and ready for deployment. Checks using the UCRT tool >> chain have been part of the CRAN check process for a while. I >> believe CRAN plans to switch R-devel checks and builds to the >> UCRT tool chain during the upcoming CRAN downtime. I expect there >> will be some communication from CRAN on this soon, including on >> any issues in supporting binaries for both R-devel and R-patched. >> >> In putting together something as large as a tool chain there will >> always be many choices, each with advantages and disadvantages. Some >> things may be advantages in some settings and not others. Taking just >> one case in point: Cross compilation. This is likely to be a better >> approach for CRAN in the future and is supported by the MXE framework >> on which the new tool chain is based. >> >> The much more recent changes in rtools4 to support UCRT are at this >> point not yet as well tested as the new tool chain. Once these changes >> to rtools4 mature, and if binary compatibility can be assured, then >> having a second tool chain may be useful in some cases. But if there >> are incompatibilities then it will be up to rtools4 to keep up with >> the tool chain used by CRAN. On the other, contributing to improving >> the MXE-based tool chain may be a better investment of time. >> >> Best, >> >> luke >> >> >> ______________________________________________ >> R-devel at r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-devel >> >> >> -- >> Luke Tierney >> Ralph E. Wareham Professor of Mathematical Sciences >> University of Iowa Phone: 319-335-3386 >> Department of Statistics and Fax: 319-335-3017 >> Actuarial Science >> 241 Schaeffer Hall email: luke-tierney at uiowa.edu >> <luke-tierney at uiowa.edu> >> Iowa City, IA 52242 WWW: http://www.stat.uiowa.edu >> >> >> Thank you, Dr. Tierney. However, I am concerned about the not-insignificant >> number of us who for various reasons can only do our development on >> Windows. Rtools has been the official tool chain with which to build >> windows for the at least 20 years I have been using R (yes, a babe in the >> woods compared to most, but not a complete neophyte). Duncan and Jereoen >> have each done yeoman?s jobs in ensuring that R can be built from source on >> Windows and that packages can be developed for all OSS on Windows?even >> Solaris SPARC. >> >> I am much less aware of Thomas?s work, and I?ll gladly take the blame for >> it, but I haven?t seen an accessible tool chain built by him which would >> allow me, the Windows developer, to build R and all packages from source on >> a native Windows box. Have I just missed it? If not, is that planned? If >> R-core switches the official Windows toolchain, where does that leave us? >> >> Thank you, >> >> Avi >> >> -- >> Sent from Gmail Mobile >> >> > Thank you, Simon. That was valuable. Skimming that quickly, I get a bit > concerned. I?ve been building from source and then using OpenBLAS in my R > source for many, many years now, and it looks like its support is tenuous > in the experimental chain. Similarly with packages like nloptr, where I > build NLOPT 2.6+ and have adjusted Jelmer?s code for it to work in R for > Windows. I maintain packages with Fortran/OpenMP and Rcpp(Parallel). I hope > that should the decision be made to switch, it will be done when the build > process is more streamlined, especially for some fundamental packages.Hi Avi, if your R package includes source code (C/C++/Fortran), there is no fundamental difference, just the version of the compilers and/or external libraries may have changed. nloptr checks are fine, see https://cran.r-project.org/web/checks/check_results_nloptr.html The architecture is r-devel-windows-x86_64-gcc10-UCRT. Best Tomas> > That being said, I must take the opportunity again to thank R-core, Tomas, > Jeroen, Duncan among many others who have built an infrastructure that > allows amateur programmers to contribute to the statistical infrastructure. > > Thanks again, > > Avi
Dirk Eddelbuettel
2021-Aug-23 23:00 UTC
[Rd] [External] Re: Update on rtools4 and ucrt support
As I type this, we are eight messages into this thread -- but I am not sure it has been made clear what the actual contentious issues are. There appear to be two toolchains, and they appear to be interoperate (though Duncan stated he had issues with an (arguably demanding) package). Now, I have the opposite (hence positive) experience. For one package I look after, a colleague took care of the (complicated in that case) 'needed to build the package' pre-requirements by ensuring we have a UCRT variant. Jeroen then (unprompted) supplied a two-line/two-file PR to enable a Windows UCRT build (piggy-backing on the existing Windows build), and with that the 'ERROR' I had at CRAN reports under Tomas UCRT entry is gone. Net-net, this looks like a working setup to me which combines both toolchains without issues. And I was able to repeat this with a few more packages of mine for which Jeroen's winlibs factory has libraries---these now build to under Tomas's builder at CRAN. So maybe this is not an either-or discussion? So if there are issues, could we be told what they are, and could we possibly help Jeroen and Tomas to iron them out? Dirk -- https://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org