R devel - Mar 2015 - Notes on building a gcc toolchain for Rtools (but not multilib)

On 10/03/2015 3:17 PM, Duncan Murdoch wrote:
> On 10/03/2015 2:56 PM, Dan Tenenbaum wrote:
> >
> >
> > ----- Original Message -----
> >> From: "Duncan Murdoch" <murdoch.duncan at
gmail.com>
> >> To: "Dan Tenenbaum" <dtenenba at fredhutch.org>
> >> Cc: "Hsiu-Khuern Tang" <tangoh at gmail.com>,
r-devel at r-project.org
> >> Sent: Tuesday, March 10, 2015 11:37:12 AM
> >> Subject: Re: [Rd] Notes on building a gcc toolchain for Rtools
(but not multilib)
> >>
> >> On 10/03/2015 12:54 PM, Dan Tenenbaum wrote:
> >>>
> >>> ----- Original Message -----
> >>>> From: "Duncan Murdoch" <murdoch.duncan at
gmail.com>
> >>>> To: "Hsiu-Khuern Tang" <tangoh at
gmail.com>, r-devel at r-project.org
> >>>> Sent: Monday, March 9, 2015 10:40:02 AM
> >>>> Subject: Re: [Rd] Notes on building a gcc toolchain for
Rtools
> >>>> (but not	multilib)
> >>>>
> >>>> On 09/03/2015 11:07 AM, Hsiu-Khuern Tang wrote:
> >>>>> On Mon, Mar 9, 2015 at 3:50 AM, Duncan Murdoch
> >>>>> <murdoch.duncan at gmail.com> wrote:
> >>>>>> On 08/03/2015 10:02 PM, Hsiu-Khuern Tang wrote:
> >>>>>>> Hi,
> >>>>>>>
> >>>>>>> [This is a follow-up to the "New version
of Rtools for
> >>>>>>> Windows"
> >>>>>>> thread
> >>>>>>> in January, but I just subscribed and
don't know how to
> >>>>>>> reply to
> >>>>>>> an
> >>>>>>> old thread -- my apologies.]
> >>>>>>
> >>>>>> I am planning to put a new Rtools online today
that uses a
> >>>>>> different
> >>>>>> build of gcc 4.9.2.  I will be concentrating on
getting it to
> >>>>>> work with
> >>>>>> all the external libraries before the 3.2.0
release next
> >>>>>> month.
> >>>>>>  I'm not
> >>>>>> planning to try to get it to work with R-patched,
and I
> >>>>>> expect it
> >>>>>> won't:
> >>>>>>  I needed to make a number of patches to R-devel
for
> >>>>>>  compatibility.
> >>>>>
> >>>>> I also worked off R-devel (I said wrongly that it was
R-patched
> >>>>> in
> >>>>> my
> >>>>> original post) and benefited from your compatibility
changes.
> >>>>>
> >>>>> I look forward to the new Rtools and will test it by
compiling
> >>>>> some
> >>>>> packages.
> >>>>
> >>>> It's now on the main site at CRAN, and should
propagate to the
> >>>> mirrors
> >>>> reasonably quickly.  I'm hoping that tomorrow's
R-devel build
> >>>> will
> >>>> use
> >>>> it, but there may be some last minute problems.
> >>>>
> >>>
> >>> Thanks to you and everyone who worked on this. Is there a way
to
> >>> tell which toolchain built a given R-devel binary?
> >>> If not, can you let us know when there is one on CRAN that was
> >>> built with the new Rtools?
> >>
> >> If you look in etc/*/Makeconf, you'll see something like this:
> >>
> >> BINPREF ?= $(RTOOLS)gcc492_64/bin/
> >>
> >> hopefully from tomorrow onwards.  If today's build was with
the new
> >> toolchain, you should see a hardcoded path to where I have Rtools
> >> installed on the build machine, which isn't so helpful.  The
previous
> >> toolchain left BINPREF blank.
> >>
> >> If you want to use your own toolchain, just edit those files.  If
you
> >> want to install the standard Rtools somewhere else, set an
> >> environment
> >> variable like
> >>
> >> RTOOLS = C:/Rtools/
> >>
> >> (where the terminal / is required.)
> >>
> >
> > Thanks, that's very helpful. I also notice with the latest R-devel
binary (67969, which is built with the new Rtools according to what you say
above) that I need to do setInternet2(TRUE) before I can download from any URLs;
I see some mention of that earlier in this thread, is this intended? If so is
there a way to make this the default?
>
> That's a bug.  I haven't tracked down what's going wrong with
our
> regular code.  If I can't find that and fix it soon, I'll make
Internet2
> the default.  For now, you can do it yourself using the instructions on
> ?setInternet2.
I've found the problem now, and it will be easy to fix.  In case anyone 
else finds this thread, here's the problem:

The regular Windows internet code (not internet2) used the Winsock 
interface.  It returns different error codes than the Unix sockets 
interface.   Some error codes are ignorable (EWOULDBLOCK and 
EINPROGRESS); these are WSAEWOULDBLOCK and WSAEINPROGRESS in Winsock.  
We had code like

#ifndef EWOULDBLOCK
# define EWOULDBLOCK             WSAEWOULDBLOCK
#endif

so our code could work with the Unix macro names.  However, the new 
toolchain defines both EWOULDBLOCK
and WSAEWOULDBLOCK, so the remapping never happened, and we didn't 
ignore those errors.

Tomorrow's R-devel should have the fix in place, unless something else 
goes wrong, or I'm too slow.

Duncan Murdoch

----- Original Message -----
> From: "Duncan Murdoch" <murdoch.duncan at gmail.com>
> To: "Dan Tenenbaum" <dtenenba at fredhutch.org>
> Cc: r-devel at r-project.org
> Sent: Wednesday, March 11, 2015 12:06:48 PM
> Subject: Re: [Rd] Notes on building a gcc toolchain for Rtools (but not
multilib)
> 
> On 10/03/2015 3:17 PM, Duncan Murdoch wrote:
> > On 10/03/2015 2:56 PM, Dan Tenenbaum wrote:
> > >
> > >
> > > ----- Original Message -----
> > >> From: "Duncan Murdoch" <murdoch.duncan at
gmail.com>
> > >> To: "Dan Tenenbaum" <dtenenba at
fredhutch.org>
> > >> Cc: "Hsiu-Khuern Tang" <tangoh at gmail.com>,
r-devel at r-project.org
> > >> Sent: Tuesday, March 10, 2015 11:37:12 AM
> > >> Subject: Re: [Rd] Notes on building a gcc toolchain for
Rtools
> > >> (but not multilib)
> > >>
> > >> On 10/03/2015 12:54 PM, Dan Tenenbaum wrote:
> > >>>
> > >>> ----- Original Message -----
> > >>>> From: "Duncan Murdoch" <murdoch.duncan
at gmail.com>
> > >>>> To: "Hsiu-Khuern Tang" <tangoh at
gmail.com>,
> > >>>> r-devel at r-project.org
> > >>>> Sent: Monday, March 9, 2015 10:40:02 AM
> > >>>> Subject: Re: [Rd] Notes on building a gcc toolchain
for Rtools
> > >>>> (but not	multilib)
> > >>>>
> > >>>> On 09/03/2015 11:07 AM, Hsiu-Khuern Tang wrote:
> > >>>>> On Mon, Mar 9, 2015 at 3:50 AM, Duncan Murdoch
> > >>>>> <murdoch.duncan at gmail.com> wrote:
> > >>>>>> On 08/03/2015 10:02 PM, Hsiu-Khuern Tang
wrote:
> > >>>>>>> Hi,
> > >>>>>>>
> > >>>>>>> [This is a follow-up to the "New
version of Rtools for
> > >>>>>>> Windows"
> > >>>>>>> thread
> > >>>>>>> in January, but I just subscribed and
don't know how to
> > >>>>>>> reply to
> > >>>>>>> an
> > >>>>>>> old thread -- my apologies.]
> > >>>>>>
> > >>>>>> I am planning to put a new Rtools online
today that uses a
> > >>>>>> different
> > >>>>>> build of gcc 4.9.2.  I will be concentrating
on getting it
> > >>>>>> to
> > >>>>>> work with
> > >>>>>> all the external libraries before the 3.2.0
release next
> > >>>>>> month.
> > >>>>>>  I'm not
> > >>>>>> planning to try to get it to work with
R-patched, and I
> > >>>>>> expect it
> > >>>>>> won't:
> > >>>>>>  I needed to make a number of patches to
R-devel for
> > >>>>>>  compatibility.
> > >>>>>
> > >>>>> I also worked off R-devel (I said wrongly that it
was
> > >>>>> R-patched
> > >>>>> in
> > >>>>> my
> > >>>>> original post) and benefited from your
compatibility changes.
> > >>>>>
> > >>>>> I look forward to the new Rtools and will test it
by
> > >>>>> compiling
> > >>>>> some
> > >>>>> packages.
> > >>>>
> > >>>> It's now on the main site at CRAN, and should
propagate to the
> > >>>> mirrors
> > >>>> reasonably quickly.  I'm hoping that
tomorrow's R-devel build
> > >>>> will
> > >>>> use
> > >>>> it, but there may be some last minute problems.
> > >>>>
> > >>>
> > >>> Thanks to you and everyone who worked on this. Is there a
way
> > >>> to
> > >>> tell which toolchain built a given R-devel binary?
> > >>> If not, can you let us know when there is one on CRAN
that was
> > >>> built with the new Rtools?
> > >>
> > >> If you look in etc/*/Makeconf, you'll see something like
this:
> > >>
> > >> BINPREF ?= $(RTOOLS)gcc492_64/bin/
> > >>
> > >> hopefully from tomorrow onwards.  If today's build was
with the
> > >> new
> > >> toolchain, you should see a hardcoded path to where I have
> > >> Rtools
> > >> installed on the build machine, which isn't so helpful. 
The
> > >> previous
> > >> toolchain left BINPREF blank.
> > >>
> > >> If you want to use your own toolchain, just edit those files.
> > >>  If you
> > >> want to install the standard Rtools somewhere else, set an
> > >> environment
> > >> variable like
> > >>
> > >> RTOOLS = C:/Rtools/
> > >>
> > >> (where the terminal / is required.)
> > >>
> > >
> > > Thanks, that's very helpful. I also notice with the latest
> > > R-devel binary (67969, which is built with the new Rtools
> > > according to what you say above) that I need to do
> > > setInternet2(TRUE) before I can download from any URLs; I see
> > > some mention of that earlier in this thread, is this intended?
> > > If so is there a way to make this the default?
> >
> > That's a bug.  I haven't tracked down what's going wrong
with our
> > regular code.  If I can't find that and fix it soon, I'll make
> > Internet2
> > the default.  For now, you can do it yourself using the
> > instructions on
> > ?setInternet2.
> 
> I've found the problem now, and it will be easy to fix.  In case
> anyone
> else finds this thread, here's the problem:
> 
> The regular Windows internet code (not internet2) used the Winsock
> interface.  It returns different error codes than the Unix sockets
> interface.   Some error codes are ignorable (EWOULDBLOCK and
> EINPROGRESS); these are WSAEWOULDBLOCK and WSAEINPROGRESS in Winsock.
> We had code like
> 
> #ifndef EWOULDBLOCK
> # define EWOULDBLOCK             WSAEWOULDBLOCK
> #endif
> 
> so our code could work with the Unix macro names.  However, the new
> toolchain defines both EWOULDBLOCK
> and WSAEWOULDBLOCK, so the remapping never happened, and we didn't
> ignore those errors.
> 
> Tomorrow's R-devel should have the fix in place, unless something
> else
> goes wrong, or I'm too slow.
> 
Thanks! Will there be a corresponding update to Rtools as well, or is that not
necessary?

Dan

On 11/03/2015 3:09 PM, Dan Tenenbaum wrote:
>
> ----- Original Message -----
> > From: "Duncan Murdoch" <murdoch.duncan at gmail.com>
> > To: "Dan Tenenbaum" <dtenenba at fredhutch.org>
> > Cc: r-devel at r-project.org
> > Sent: Wednesday, March 11, 2015 12:06:48 PM
> > Subject: Re: [Rd] Notes on building a gcc toolchain for Rtools (but
not multilib)
> >
> > On 10/03/2015 3:17 PM, Duncan Murdoch wrote:
> > > On 10/03/2015 2:56 PM, Dan Tenenbaum wrote:
> > > >
> > > >
> > > > ----- Original Message -----
> > > >> From: "Duncan Murdoch" <murdoch.duncan at
gmail.com>
> > > >> To: "Dan Tenenbaum" <dtenenba at
fredhutch.org>
> > > >> Cc: "Hsiu-Khuern Tang" <tangoh at
gmail.com>, r-devel at r-project.org
> > > >> Sent: Tuesday, March 10, 2015 11:37:12 AM
> > > >> Subject: Re: [Rd] Notes on building a gcc toolchain for
Rtools
> > > >> (but not multilib)
> > > >>
> > > >> On 10/03/2015 12:54 PM, Dan Tenenbaum wrote:
> > > >>>
> > > >>> ----- Original Message -----
> > > >>>> From: "Duncan Murdoch"
<murdoch.duncan at gmail.com>
> > > >>>> To: "Hsiu-Khuern Tang" <tangoh at
gmail.com>,
> > > >>>> r-devel at r-project.org
> > > >>>> Sent: Monday, March 9, 2015 10:40:02 AM
> > > >>>> Subject: Re: [Rd] Notes on building a gcc
toolchain for Rtools
> > > >>>> (but not	multilib)
> > > >>>>
> > > >>>> On 09/03/2015 11:07 AM, Hsiu-Khuern Tang wrote:
> > > >>>>> On Mon, Mar 9, 2015 at 3:50 AM, Duncan
Murdoch
> > > >>>>> <murdoch.duncan at gmail.com> wrote:
> > > >>>>>> On 08/03/2015 10:02 PM, Hsiu-Khuern Tang
wrote:
> > > >>>>>>> Hi,
> > > >>>>>>>
> > > >>>>>>> [This is a follow-up to the
"New version of Rtools for
> > > >>>>>>> Windows"
> > > >>>>>>> thread
> > > >>>>>>> in January, but I just subscribed
and don't know how to
> > > >>>>>>> reply to
> > > >>>>>>> an
> > > >>>>>>> old thread -- my apologies.]
> > > >>>>>>
> > > >>>>>> I am planning to put a new Rtools online
today that uses a
> > > >>>>>> different
> > > >>>>>> build of gcc 4.9.2.  I will be
concentrating on getting it
> > > >>>>>> to
> > > >>>>>> work with
> > > >>>>>> all the external libraries before the
3.2.0 release next
> > > >>>>>> month.
> > > >>>>>>  I'm not
> > > >>>>>> planning to try to get it to work with
R-patched, and I
> > > >>>>>> expect it
> > > >>>>>> won't:
> > > >>>>>>  I needed to make a number of patches to
R-devel for
> > > >>>>>>  compatibility.
> > > >>>>>
> > > >>>>> I also worked off R-devel (I said wrongly
that it was
> > > >>>>> R-patched
> > > >>>>> in
> > > >>>>> my
> > > >>>>> original post) and benefited from your
compatibility changes.
> > > >>>>>
> > > >>>>> I look forward to the new Rtools and will
test it by
> > > >>>>> compiling
> > > >>>>> some
> > > >>>>> packages.
> > > >>>>
> > > >>>> It's now on the main site at CRAN, and
should propagate to the
> > > >>>> mirrors
> > > >>>> reasonably quickly.  I'm hoping that
tomorrow's R-devel build
> > > >>>> will
> > > >>>> use
> > > >>>> it, but there may be some last minute problems.
> > > >>>>
> > > >>>
> > > >>> Thanks to you and everyone who worked on this. Is
there a way
> > > >>> to
> > > >>> tell which toolchain built a given R-devel binary?
> > > >>> If not, can you let us know when there is one on
CRAN that was
> > > >>> built with the new Rtools?
> > > >>
> > > >> If you look in etc/*/Makeconf, you'll see something
like this:
> > > >>
> > > >> BINPREF ?= $(RTOOLS)gcc492_64/bin/
> > > >>
> > > >> hopefully from tomorrow onwards.  If today's build
was with the
> > > >> new
> > > >> toolchain, you should see a hardcoded path to where I
have
> > > >> Rtools
> > > >> installed on the build machine, which isn't so
helpful.  The
> > > >> previous
> > > >> toolchain left BINPREF blank.
> > > >>
> > > >> If you want to use your own toolchain, just edit those
files.
> > > >>  If you
> > > >> want to install the standard Rtools somewhere else, set
an
> > > >> environment
> > > >> variable like
> > > >>
> > > >> RTOOLS = C:/Rtools/
> > > >>
> > > >> (where the terminal / is required.)
> > > >>
> > > >
> > > > Thanks, that's very helpful. I also notice with the
latest
> > > > R-devel binary (67969, which is built with the new Rtools
> > > > according to what you say above) that I need to do
> > > > setInternet2(TRUE) before I can download from any URLs; I
see
> > > > some mention of that earlier in this thread, is this
intended?
> > > > If so is there a way to make this the default?
> > >
> > > That's a bug.  I haven't tracked down what's going
wrong with our
> > > regular code.  If I can't find that and fix it soon, I'll
make
> > > Internet2
> > > the default.  For now, you can do it yourself using the
> > > instructions on
> > > ?setInternet2.
> >
> > I've found the problem now, and it will be easy to fix.  In case
> > anyone
> > else finds this thread, here's the problem:
> >
> > The regular Windows internet code (not internet2) used the Winsock
> > interface.  It returns different error codes than the Unix sockets
> > interface.   Some error codes are ignorable (EWOULDBLOCK and
> > EINPROGRESS); these are WSAEWOULDBLOCK and WSAEINPROGRESS in Winsock.
> > We had code like
> >
> > #ifndef EWOULDBLOCK
> > # define EWOULDBLOCK             WSAEWOULDBLOCK
> > #endif
> >
> > so our code could work with the Unix macro names.  However, the new
> > toolchain defines both EWOULDBLOCK
> > and WSAEWOULDBLOCK, so the remapping never happened, and we didn't
> > ignore those errors.
> >
> > Tomorrow's R-devel should have the fix in place, unless something
> > else
> > goes wrong, or I'm too slow.
> >
>
> Thanks! Will there be a corresponding update to Rtools as well, or is that
not necessary?
Not necessary.

Duncan Murdoch