R SIG Debian - Jun 2021 - LTO flags in 4.1.0 Ubuntu 21.04 Makeconf

Hi Karl,

On 1 June 2021 at 19:12, Karl Dunkle Werner wrote:
| Thanks for the very helpful response!

Thanks for weaving the threads back together.

| Got it, and thank you for the offer! I can edit my Makeconf (everything 
| works when I remove the LTO-related flags). I was hoping to make 
| installation smoother for other Arrow and Ubuntu users, but maybe that's 
| something users or the Arrow developers will need to tackle.

Yes indeed. One difficulty is that package authors can generally not override
what is in R's makeconf. As e.g. with my Rcpp maintainer hat on I have found
that frustrating. But such is life with R: not everything can be altered on
the fly.
 
| I think I'm confused by this comment (because I don't know much about 
| the packaging process).
| > - the Debian and hence derived Ubuntu package of R 4.1.0 does*not* 
enable
| >    LTO yet, in other words I have not yet add --enable-lto to the
configure
| >    call and we use to default of 'nope'
| I'm confused because the R 4.1.0 Debian versions have flags like:
| CXX11FLAGS = -g -O2 
| -ffile-prefix-map=/home/jranke/git/r-backports/bullseye/r-base-4.1.0=. 
| -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time 
| -D_FORTIFY_SOURCE=2 -g $(LTO)
| 
| The R 4.1.0 Ubuntu 20.10 flags are:
| CXX11FLAGS = -g -O2 
| -fdebug-prefix-map=/build/r-base-2Aokdc/r-base-4.1.0=. 
| -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time 
| -D_FORTIFY_SOURCE=2 -g $(LTO)
| 
| While the R 4.1.0 Ubuntu 21.04 flags are:
| CXX11FLAGS = -g -O2 
| -ffile-prefix-map=/build/r-base-aXXzqd/r-base-4.1.0=. -flto=auto 
| -ffat-lto-objects -fstack-protector-strong -Wformat 
| -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g $(LTO)
| 
| Should the Ubuntu 21.04 LTO flags be different than the others?

<shrug emoji here>

Bbut these come from two (or three) distinct sources
- default compiler settings plus distro settings (really two players)
- package changes

and here you only one of the three constant (R 4.1.0). I think the -flto=auto
(amd -ffat-lto-objects) may come from gcc itself.  I can assure you (and you
can test that by looking at the git log at salsa.debian.org under my
'edd'
handle for the r-base package) that the debian/rules files configuring the
build _has not changed_ recently and does not set either.  In particularly no
with/without lto settings.  It's all 'without lto by default' from
my end. So
back to ... distribution and compiler changes. And I have learned in the 20+
years here to trust my colleagues looking after gcc et al.

Also, and just to be plain, recall that $(LTO) is calling a variable which
may, or may not, bet set.  When I look at my most recent package build logs
I see the compilation (in the long log) commence with

make[4]: Entering directory '/build/r-base-4.1.0/src/main'
gcc -std=gnu99 -I../../src/extra  -I. -I../../src/include -I../../src/include  
-I../../src/nmath -DHAVE_CONFIG_H   -fopenmp -fpic  -g -O2
-ffile-prefix-map=/build/r-base-4.1.0=. -fstack-protector-strong -Wformat
-Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c CommandLineArgs.c
-o CommandLineArgs.o

No LTO or -flto here (or in the rest of the file). I may of course have
missed something.

| My challenges are with the Makeconf flags packaged with Ubuntu 21.04's R 
| 4.1.0, but the example above also fails when the same Makeconf file is 
| copied into the rocker/r-base:4.0.5 image.

I have not looked as deeply into this as you but I can assure that I myself
of the maintainer of 'r-base' have been deeply and profoundly frustrated
by
the inability of this one package to build properly.  If you can improve that
without heavy surgery, side effects to other packages, or other shenanigans I
would be all ears.

Cheers, Dirk

-- 
https://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org

Hello Dirk,

I see, this is all very helpful. Thank you!


For anyone else who runs into this issue, you can do either of the 
following;
(a) Remove the flags "-flto=auto" and "-ffat-lto-objects"
from the
CXX11FLAGS in your /etc/R/Makeconf
(b) Set CXX11FLAGS in your Makevars (in $HOME/.R/Makevars) to something 
like the following:

CXX11FLAGS = -g -O2 -ffile-prefix-map=/build/r-base-aXXzqd/r-base-4.1.0=.
-fstack-protector-strong -Wformat -Werror=format-security -Wdate-time
-D_FORTIFY_SOURCE=2 -g

(This is just the Makeconf CXX11FLAGS with LTO flags removed).
Note that these may turn off LTO for other packages you install, 
depending on how they manage build flags.


Thanks,
Karl



On 6/1/21 7:48 PM, Dirk Eddelbuettel wrote:
> Hi Karl,
>
> On 1 June 2021 at 19:12, Karl Dunkle Werner wrote:
> | Thanks for the very helpful response!
>
> Thanks for weaving the threads back together.
>
> | Got it, and thank you for the offer! I can edit my Makeconf (everything
> | works when I remove the LTO-related flags). I was hoping to make
> | installation smoother for other Arrow and Ubuntu users, but maybe
that's
> | something users or the Arrow developers will need to tackle.
>
> Yes indeed. One difficulty is that package authors can generally not
override
> what is in R's makeconf. As e.g. with my Rcpp maintainer hat on I have
found
> that frustrating. But such is life with R: not everything can be altered on
> the fly.
>   
> | I think I'm confused by this comment (because I don't know much
about
> | the packaging process).
> | > - the Debian and hence derived Ubuntu package of R 4.1.0 does*not* 
enable
> | >    LTO yet, in other words I have not yet add --enable-lto to the
configure
> | >    call and we use to default of 'nope'
> | I'm confused because the R 4.1.0 Debian versions have flags like:
> | CXX11FLAGS = -g -O2
> | -ffile-prefix-map=/home/jranke/git/r-backports/bullseye/r-base-4.1.0=.
> | -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time
> | -D_FORTIFY_SOURCE=2 -g $(LTO)
> |
> | The R 4.1.0 Ubuntu 20.10 flags are:
> | CXX11FLAGS = -g -O2
> | -fdebug-prefix-map=/build/r-base-2Aokdc/r-base-4.1.0=.
> | -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time
> | -D_FORTIFY_SOURCE=2 -g $(LTO)
> |
> | While the R 4.1.0 Ubuntu 21.04 flags are:
> | CXX11FLAGS = -g -O2
> | -ffile-prefix-map=/build/r-base-aXXzqd/r-base-4.1.0=. -flto=auto
> | -ffat-lto-objects -fstack-protector-strong -Wformat
> | -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g $(LTO)
> |
> | Should the Ubuntu 21.04 LTO flags be different than the others?
>
> <shrug emoji here>
>
> Bbut these come from two (or three) distinct sources
> - default compiler settings plus distro settings (really two players)
> - package changes
>
> and here you only one of the three constant (R 4.1.0). I think the
-flto=auto
> (amd -ffat-lto-objects) may come from gcc itself.  I can assure you (and
you
> can test that by looking at the git log at salsa.debian.org under my
'edd'
> handle for the r-base package) that the debian/rules files configuring the
> build _has not changed_ recently and does not set either.  In particularly
no
> with/without lto settings.  It's all 'without lto by default'
from my end. So
> back to ... distribution and compiler changes. And I have learned in the
20+
> years here to trust my colleagues looking after gcc et al.
>
> Also, and just to be plain, recall that $(LTO) is calling a variable which
> may, or may not, bet set.  When I look at my most recent package build logs
> I see the compilation (in the long log) commence with
>
> make[4]: Entering directory '/build/r-base-4.1.0/src/main'
> gcc -std=gnu99 -I../../src/extra  -I. -I../../src/include
-I../../src/include   -I../../src/nmath -DHAVE_CONFIG_H   -fopenmp -fpic  -g -O2
-ffile-prefix-map=/build/r-base-4.1.0=. -fstack-protector-strong -Wformat
-Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c CommandLineArgs.c
-o CommandLineArgs.o
>
> No LTO or -flto here (or in the rest of the file). I may of course have
> missed something.
>
> | My challenges are with the Makeconf flags packaged with Ubuntu
21.04's R
> | 4.1.0, but the example above also fails when the same Makeconf file is
> | copied into the rocker/r-base:4.0.5 image.
>
> I have not looked as deeply into this as you but I can assure that I myself
> of the maintainer of 'r-base' have been deeply and profoundly
frustrated by
> the inability of this one package to build properly.  If you can improve
that
> without heavy surgery, side effects to other packages, or other shenanigans
I
> would be all ears.
>
> Cheers, Dirk
>