R devel - Mar 2015 - Notes on building a gcc toolchain for Rtools (but not multilib)

On 11/03/2015 3:09 PM, Dan Tenenbaum wrote:
>
> ----- Original Message -----
> > From: "Duncan Murdoch" <murdoch.duncan at gmail.com>
> > To: "Dan Tenenbaum" <dtenenba at fredhutch.org>
> > Cc: r-devel at r-project.org
> > Sent: Wednesday, March 11, 2015 12:06:48 PM
> > Subject: Re: [Rd] Notes on building a gcc toolchain for Rtools (but
not multilib)
> >
> > On 10/03/2015 3:17 PM, Duncan Murdoch wrote:
> > > On 10/03/2015 2:56 PM, Dan Tenenbaum wrote:
> > > >
> > > >
> > > > ----- Original Message -----
> > > >> From: "Duncan Murdoch" <murdoch.duncan at
gmail.com>
> > > >> To: "Dan Tenenbaum" <dtenenba at
fredhutch.org>
> > > >> Cc: "Hsiu-Khuern Tang" <tangoh at
gmail.com>, r-devel at r-project.org
> > > >> Sent: Tuesday, March 10, 2015 11:37:12 AM
> > > >> Subject: Re: [Rd] Notes on building a gcc toolchain for
Rtools
> > > >> (but not multilib)
> > > >>
> > > >> On 10/03/2015 12:54 PM, Dan Tenenbaum wrote:
> > > >>>
> > > >>> ----- Original Message -----
> > > >>>> From: "Duncan Murdoch"
<murdoch.duncan at gmail.com>
> > > >>>> To: "Hsiu-Khuern Tang" <tangoh at
gmail.com>,
> > > >>>> r-devel at r-project.org
> > > >>>> Sent: Monday, March 9, 2015 10:40:02 AM
> > > >>>> Subject: Re: [Rd] Notes on building a gcc
toolchain for Rtools
> > > >>>> (but not	multilib)
> > > >>>>
> > > >>>> On 09/03/2015 11:07 AM, Hsiu-Khuern Tang wrote:
> > > >>>>> On Mon, Mar 9, 2015 at 3:50 AM, Duncan
Murdoch
> > > >>>>> <murdoch.duncan at gmail.com> wrote:
> > > >>>>>> On 08/03/2015 10:02 PM, Hsiu-Khuern Tang
wrote:
> > > >>>>>>> Hi,
> > > >>>>>>>
> > > >>>>>>> [This is a follow-up to the
"New version of Rtools for
> > > >>>>>>> Windows"
> > > >>>>>>> thread
> > > >>>>>>> in January, but I just subscribed
and don't know how to
> > > >>>>>>> reply to
> > > >>>>>>> an
> > > >>>>>>> old thread -- my apologies.]
> > > >>>>>>
> > > >>>>>> I am planning to put a new Rtools online
today that uses a
> > > >>>>>> different
> > > >>>>>> build of gcc 4.9.2.  I will be
concentrating on getting it
> > > >>>>>> to
> > > >>>>>> work with
> > > >>>>>> all the external libraries before the
3.2.0 release next
> > > >>>>>> month.
> > > >>>>>>  I'm not
> > > >>>>>> planning to try to get it to work with
R-patched, and I
> > > >>>>>> expect it
> > > >>>>>> won't:
> > > >>>>>>  I needed to make a number of patches to
R-devel for
> > > >>>>>>  compatibility.
> > > >>>>>
> > > >>>>> I also worked off R-devel (I said wrongly
that it was
> > > >>>>> R-patched
> > > >>>>> in
> > > >>>>> my
> > > >>>>> original post) and benefited from your
compatibility changes.
> > > >>>>>
> > > >>>>> I look forward to the new Rtools and will
test it by
> > > >>>>> compiling
> > > >>>>> some
> > > >>>>> packages.
> > > >>>>
> > > >>>> It's now on the main site at CRAN, and
should propagate to the
> > > >>>> mirrors
> > > >>>> reasonably quickly.  I'm hoping that
tomorrow's R-devel build
> > > >>>> will
> > > >>>> use
> > > >>>> it, but there may be some last minute problems.
> > > >>>>
> > > >>>
> > > >>> Thanks to you and everyone who worked on this. Is
there a way
> > > >>> to
> > > >>> tell which toolchain built a given R-devel binary?
> > > >>> If not, can you let us know when there is one on
CRAN that was
> > > >>> built with the new Rtools?
> > > >>
> > > >> If you look in etc/*/Makeconf, you'll see something
like this:
> > > >>
> > > >> BINPREF ?= $(RTOOLS)gcc492_64/bin/
> > > >>
> > > >> hopefully from tomorrow onwards.  If today's build
was with the
> > > >> new
> > > >> toolchain, you should see a hardcoded path to where I
have
> > > >> Rtools
> > > >> installed on the build machine, which isn't so
helpful.  The
> > > >> previous
> > > >> toolchain left BINPREF blank.
> > > >>
> > > >> If you want to use your own toolchain, just edit those
files.
> > > >>  If you
> > > >> want to install the standard Rtools somewhere else, set
an
> > > >> environment
> > > >> variable like
> > > >>
> > > >> RTOOLS = C:/Rtools/
> > > >>
> > > >> (where the terminal / is required.)
> > > >>
> > > >
> > > > Thanks, that's very helpful. I also notice with the
latest
> > > > R-devel binary (67969, which is built with the new Rtools
> > > > according to what you say above) that I need to do
> > > > setInternet2(TRUE) before I can download from any URLs; I
see
> > > > some mention of that earlier in this thread, is this
intended?
> > > > If so is there a way to make this the default?
> > >
> > > That's a bug.  I haven't tracked down what's going
wrong with our
> > > regular code.  If I can't find that and fix it soon, I'll
make
> > > Internet2
> > > the default.  For now, you can do it yourself using the
> > > instructions on
> > > ?setInternet2.
> >
> > I've found the problem now, and it will be easy to fix.  In case
> > anyone
> > else finds this thread, here's the problem:
> >
> > The regular Windows internet code (not internet2) used the Winsock
> > interface.  It returns different error codes than the Unix sockets
> > interface.   Some error codes are ignorable (EWOULDBLOCK and
> > EINPROGRESS); these are WSAEWOULDBLOCK and WSAEINPROGRESS in Winsock.
> > We had code like
> >
> > #ifndef EWOULDBLOCK
> > # define EWOULDBLOCK             WSAEWOULDBLOCK
> > #endif
> >
> > so our code could work with the Unix macro names.  However, the new
> > toolchain defines both EWOULDBLOCK
> > and WSAEWOULDBLOCK, so the remapping never happened, and we didn't
> > ignore those errors.
> >
> > Tomorrow's R-devel should have the fix in place, unless something
> > else
> > goes wrong, or I'm too slow.
> >
>
> Thanks! Will there be a corresponding update to Rtools as well, or is that
not necessary?
Not necessary.

Duncan Murdoch

I've just uploaded a minor update (3.3.0.1957) to Rtools33, adding the 
cygpath.exe utility.  That utility converts between Windows style paths 
like D:/Rtools and Cygwin style paths like /cygdrive/d/Rtools.   It may 
be useful in configuration files if your external library expects to 
find gcc on the path, since Rtools no longer puts it there.  Assuming 
you want to use the Rtools toolchain, you can construct the path to the 
gcc directory in your Makevars.win file as

$(cygpath $(RTOOLS))gcc492_$(WIN)/bin

(where RTOOLS and WIN are macros from RHOME/etc/*/Makeconf that should 
already have been read.)

Thanks to JJ Allaire for the prompting on this.

Duncan Murdoch

----- Original Message -----
> From: "Duncan Murdoch" <murdoch.duncan at gmail.com>
> Cc: r-devel at r-project.org
> Sent: Thursday, March 12, 2015 10:17:23 AM
> Subject: Re: [Rd] Notes on building a gcc toolchain for Rtools (but not
multilib)
> 
> I've just uploaded a minor update (3.3.0.1957) to Rtools33, adding
> the
> cygpath.exe utility.  That utility converts between Windows style
> paths
> like D:/Rtools and Cygwin style paths like /cygdrive/d/Rtools.   It
> may
> be useful in configuration files if your external library expects to
> find gcc on the path, since Rtools no longer puts it there.  Assuming
> you want to use the Rtools toolchain, you can construct the path to
> the
> gcc directory in your Makevars.win file as
> 
> $(cygpath $(RTOOLS))gcc492_$(WIN)/bin
> 
> (where RTOOLS and WIN are macros from RHOME/etc/*/Makeconf that
> should
> already have been read.)
> 
I had some problems with this. In the Bioconductor package zlibbioc, for example
(which wraps zlib) there is a lower-level "Makefile.gcc" inside
src/zlib-1.2.5/win32 which contains:

PREFIX ifeq "$(WIN)" "64"
CC = $(PREFIX)gcc -m64
else
CC = $(PREFIX)gcc -m32
endif


First of all, even though I have cygpath installed, using it as you suggest does
not work, and does not seem to be necessary. Secondly, although $(WIN) is
visible to the ifeq statement, it can't be expanded, that is to say,
changing PREFIX to

PREFIX = "$(RTOOLS)gcc492_$(WIN)/bin/"

doesn't work, and the error shows that gcc is being looked for in a path
that ends with gcc492_/bin/ , in other words, the $(WIN) does not expand.
However, changing to this:

ifeq "$(WIN)" "64"
PREFIX = "$(RTOOLS)gcc492_64/bin/"
CC = $(PREFIX)gcc -m64
else
PREFIX = "$(RTOOLS)gcc492_32/bin/"
CC = $(PREFIX)gcc -m32
endif

Works just fine. It's odd that the value of $(WIN) can be "seen"
in order to correctly execute
the "ifeq", but it can't be expanded.

There are a number of fairly important packages in Bioconductor (and in CRAN too
I imagine) that wrap C/C++ libraries and they all probably need some tweaking
like this. I'm a little concerned that it's not such good practice to
change third party code in this way, it makes maintenance just a little harder.
It would be nice if maintainers could just add the latest version of the library
to their packages without changing anything. But if making these changes is the
only option, then so be it.

Dan


> Thanks to JJ Allaire for the prompting on this.
> 
> Duncan Murdoch
> 
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>