Mark Dalphin
2016-Apr-20 22:50 UTC
[Rd] Fresh build from source of R-3.2.5 failing "make check" under 64-bit Ubuntu
Hi, Let me justify building R from source. While I can obtain R built for me under Ubuntu, I tend to build it from scratch as we use a NFS-shared build as well as shared R libraries for all the bioinfo staff at this company. Only one build and one set of packages to ensure uniformity across all workstations. My problem is that despite using a pretty standard build process, I am failing at the "make check" step. I don't see what I might have done that is causing the failure. My platform is:> uname -aLinux littlebourne 3.13.0-85-generic #129-Ubuntu SMP Thu Mar 17 20:50:15 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux> cat /etc/issueUbuntu 14.04.4 LTS \n \l I've downloaded the recent R-3.2.5 tar-gz file and gone through the standard build, though in a non-standard location. I keep copies of the tar-gzipped file in an "Archive" directory.> gunzip -c Archive/R-3.2.5.tar.gz | tar xf - > cd R-3.2.5/ > ./configure --prefix=/opt/apps/x86_64/R/R-3.2.5 LIBnn=lib[ ... much output ...] R is now configured for x86_64-pc-linux-gnu Source directory: . Installation directory: /opt/apps/x86_64/R/R-3.2.5 C compiler: gcc -std=gnu99 -g -O2 Fortran 77 compiler: gfortran -g -O2 C++ compiler: g++ -g -O2 C++ 11 compiler: g++ -std=c++11 -g -O2 Fortran 90/95 compiler: gfortran -g -O2 Obj-C compiler: gcc -g -O2 -fobjc-exceptions Interfaces supported: X11, tcltk External libraries: readline, zlib, bzlib, lzma, PCRE Additional capabilities: PNG, JPEG, TIFF, NLS, cairo Options enabled: shared BLAS, R profiling Capabilities skipped: ICU Options not enabled: memory profiling Recommended packages: yes # Note: I use tcsh, not bash> make >& make.out &# There are no obvious errors and the 'make' proceeds finally through some Java configuration # and then exits with a zero status.> make check >& make_check.out &[ ... much output captured ...] make[3]: Entering directory `/opt/apps/x86_64/R/src/R-3.2.5/tests' running code in 'reg-tests-1a.R' ... OK running code in 'reg-tests-1b.R' ... OK running code in 'reg-tests-1c.R' ... OK running code in 'reg-tests-2.R' ... OK comparing 'reg-tests-2.Rout' to './reg-tests-2.Rout.save' ... OK running code in 'reg-examples1.R' ... OK running code in 'reg-examples2.R' ... OK running code in 'reg-packages.R' ...make[3]: *** [reg-packages.Rout] Error 1 make[3]: Leaving directory `/opt/apps/x86_64/R/src/R-3.2.5/tests' make[2]: *** [test-Reg] Error 2 make[2]: Leaving directory `/opt/apps/x86_64/R/src/R-3.2.5/tests' make[1]: *** [test-all-basics] Error 1 make[1]: Leaving directory `/opt/apps/x86_64/R/src/R-3.2.5/tests' make: *** [check] Error 2> cd tests > tail -30 reg-packages.Rout.fail*** installing help indices ** building package indices ** testing if installed package can be loaded * DONE (pkgB) Loading required package: pkgB building package exNSS4 ... installing package exNSS4 using file exNSS4_1.1.tar.gz ... * installing *source* package 'exNSS4' ... ** R ** preparing package for lazy loading Creating a generic function for 'plot' from package 'graphics' in package 'exNSS4' ** help No man pages found in package 'exNSS4' *** installing help indices ** building package indices ** testing if installed package can be loaded * DONE (exNSS4) Loading required package: exNSS4> (res <- installed.packages(lib.loc = "myLib", priority = "NA"))Package LibPath Version Priority Depends Imports LinkingTo Suggests myTst "myTst" "myLib" "1.0" NA "methods" NA NA NA Enhances License License_is_FOSS myTst NA "What license is it under?" NA License_restricts_use OS_type MD5sum NeedsCompilation Built myTst NA NA NA NA "3.2.5"> stopifnot(identical(res[,"Package"], setNames(,sort(c(p.lis, "myTst")))),+ res[,"LibPath"] == "myLib") Error: identical(res[, "Package"], setNames(, sort(c(p.lis, "myTst")))) is not TRUE Execution halted> ls myLibexNSS4 myTst pkgA pkgB So, it looks like the "installed.packages()" is not correctly detecting what is present in "myLib". Has anyone else seen this? Has anyone got any ideas about what is going wrong? My environment does not include R_LIBS, LD_LIBRARY, etc. The PATH does include an older version of R, 3.1.1. Regards, Mark Dalphin
peter dalgaard
2016-Apr-21 11:20 UTC
[Rd] Fresh build from source of R-3.2.5 failing "make check" under 64-bit Ubuntu
Hmm. I don't think this has been seen by other Ubuntu users, or other Linuxen for that matter. You have a gratuitous use of "&", but I don't really see how that would give these symptoms, unless you accidentally run a command twice, creating a race condition between two background processes. More likely it is file system related, like NFS directory caching or maybe permissions, although the latter seems unlikely if one package does install properly. You might want to recheck whether installed.packages(lib.loc = "myLib", priority = "NA") gives different results a bit later, and if not, check that the package descriptions are sane. -pd On 21 Apr 2016, at 00:50 , Mark Dalphin <mark.dalphin at pacificedge.co.nz> wrote:> Hi, > > Let me justify building R from source. While I can obtain R built for me > under Ubuntu, I tend to build it from scratch as we use a NFS-shared > build as well as shared R libraries for all the bioinfo staff at this > company. Only one build and one set of packages to ensure uniformity > across all workstations. > > My problem is that despite using a pretty standard build process, I am > failing at the "make check" step. I don't see what I might have done > that is causing the failure. > > My platform is: > >> uname -a > Linux littlebourne 3.13.0-85-generic #129-Ubuntu SMP Thu Mar 17 20:50:15 > UTC 2016 x86_64 x86_64 x86_64 GNU/Linux > >> cat /etc/issue > Ubuntu 14.04.4 LTS \n \l > > I've downloaded the recent R-3.2.5 tar-gz file and gone through the > standard build, though in a non-standard location. I keep copies of the > tar-gzipped file in an "Archive" directory. > >> gunzip -c Archive/R-3.2.5.tar.gz | tar xf - >> cd R-3.2.5/ >> ./configure --prefix=/opt/apps/x86_64/R/R-3.2.5 LIBnn=lib > [ ... much output ...] > R is now configured for x86_64-pc-linux-gnu > > Source directory: . > Installation directory: /opt/apps/x86_64/R/R-3.2.5 > > C compiler: gcc -std=gnu99 -g -O2 > Fortran 77 compiler: gfortran -g -O2 > > C++ compiler: g++ -g -O2 > C++ 11 compiler: g++ -std=c++11 -g -O2 > Fortran 90/95 compiler: gfortran -g -O2 > Obj-C compiler: gcc -g -O2 -fobjc-exceptions > > Interfaces supported: X11, tcltk > External libraries: readline, zlib, bzlib, lzma, PCRE > Additional capabilities: PNG, JPEG, TIFF, NLS, cairo > Options enabled: shared BLAS, R profiling > > Capabilities skipped: ICU > Options not enabled: memory profiling > > Recommended packages: yes > > # Note: I use tcsh, not bash >> make >& make.out & > # There are no obvious errors and the 'make' proceeds finally through > some Java configuration > # and then exits with a zero status. > >> make check >& make_check.out & > [ ... much output captured ...] > make[3]: Entering directory `/opt/apps/x86_64/R/src/R-3.2.5/tests' > running code in 'reg-tests-1a.R' ... OK > running code in 'reg-tests-1b.R' ... OK > running code in 'reg-tests-1c.R' ... OK > running code in 'reg-tests-2.R' ... OK > comparing 'reg-tests-2.Rout' to './reg-tests-2.Rout.save' ... OK > running code in 'reg-examples1.R' ... OK > running code in 'reg-examples2.R' ... OK > running code in 'reg-packages.R' ...make[3]: *** [reg-packages.Rout] Error 1 > make[3]: Leaving directory `/opt/apps/x86_64/R/src/R-3.2.5/tests' > make[2]: *** [test-Reg] Error 2 > make[2]: Leaving directory `/opt/apps/x86_64/R/src/R-3.2.5/tests' > make[1]: *** [test-all-basics] Error 1 > make[1]: Leaving directory `/opt/apps/x86_64/R/src/R-3.2.5/tests' > make: *** [check] Error 2 > >> cd tests >> tail -30 reg-packages.Rout.fail > *** installing help indices > ** building package indices > ** testing if installed package can be loaded > * DONE (pkgB) > Loading required package: pkgB > building package exNSS4 ... > > installing package exNSS4 using file exNSS4_1.1.tar.gz ... > * installing *source* package 'exNSS4' ... > ** R > ** preparing package for lazy loading > Creating a generic function for 'plot' from package 'graphics' in > package 'exNSS4' > ** help > No man pages found in package 'exNSS4' > *** installing help indices > ** building package indices > ** testing if installed package can be loaded > * DONE (exNSS4) > Loading required package: exNSS4 >> (res <- installed.packages(lib.loc = "myLib", priority = "NA")) > Package LibPath Version Priority Depends Imports LinkingTo Suggests > myTst "myTst" "myLib" "1.0" NA "methods" NA NA NA > Enhances License License_is_FOSS > myTst NA "What license is it under?" NA > License_restricts_use OS_type MD5sum NeedsCompilation Built > myTst NA NA NA NA "3.2.5" >> stopifnot(identical(res[,"Package"], setNames(,sort(c(p.lis, "myTst")))), > + res[,"LibPath"] == "myLib") > Error: identical(res[, "Package"], setNames(, sort(c(p.lis, "myTst")))) > is not TRUE > Execution halted > >> ls myLib > exNSS4 myTst pkgA pkgB > > So, it looks like the "installed.packages()" is not correctly detecting > what is present in "myLib". > > Has anyone else seen this? Has anyone got any ideas about what is going > wrong? My environment does not include R_LIBS, LD_LIBRARY, etc. The PATH > does include an older version of R, 3.1.1. > > Regards, > Mark Dalphin > > ______________________________________________ > R-devel at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel-- Peter Dalgaard, Professor, Center for Statistics, Copenhagen Business School Solbjerg Plads 3, 2000 Frederiksberg, Denmark Phone: (+45)38153501 Office: A 4.23 Email: pd.mes at cbs.dk Priv: PDalgd at gmail.com
Mark Dalphin
2016-Apr-21 22:25 UTC
[Rd] Fresh build from source of R-3.2.5 failing "make check" under 64-bit Ubuntu [SOLVED]
Testing for an NFS effect on the failure of 'make check':
Try first on /usr/local/
9:42 cd /usr/local/src/
9:42 sudo mkdir R
9:43 sudo chown mdalphin:mdalphin R
9:43 cd R
9:43 gunzip -c /opt/apps/x86_64/R/Archive/R-3.2.5.tar.gz | tar xf -
9:43 cd R-3.2.5/
9:44 ./configure --prefix=/opt/apps/x86_64/R/R-3.2.5 LIBnn=lib
9:44 make
9:57 make check
This PASSES all tests. Exit status of ZERO.
Now try on NFS:
10:04 cd /opt/apps/x86_64/R/src/
10:05 gunzip -c /opt/apps/x86_64/R/Archive/R-3.2.5.tar.gz | tar xf -
10:06 cd R-3.2.5/
10:06 ./configure --prefix=/opt/apps/x86_64/R/R-3.2.5 LIBnn=lib
10:06 make
10:14 make check
This FAILS (see below). Exit status of NON-ZERO:
running code in 'reg-packages.R' ...make[3]: *** [reg-packages.Rout]
Error 1
make[3]: Leaving directory `/opt/apps/x86_64/R/src/R-3.2.5/tests'
make[2]: *** [test-Reg] Error 2
make[2]: Leaving directory `/opt/apps/x86_64/R/src/R-3.2.5/tests'
make[1]: *** [test-all-basics] Error 1
make[1]: Leaving directory `/opt/apps/x86_64/R/src/R-3.2.5/tests'
make: *** [check] Error 2
What might be different?
> ls -ld /opt/apps/x86_64/R/src/R-3.2.5/ /usr/local/src/R/R-3.2.5/
drwxr-xr-x 15 mdalphin mdalphin 4096 Apr 22 09:57
/opt/apps/x86_64/R/src/R-3.2.5/
drwxr-xr-x 15 mdalphin mdalphin 4096 Apr 22 09:50 /usr/local/src/R/R-3.2.5/
> mount | grep /opt/
myHost.science:/mnt/home/opt/apps on /opt/apps type nfs
(rw,soft,bg,nfsvers=3,addr=XXX.XXX.XXX.XXX)
Just the NFS, I guess. This is not good, but not an R-Devel problem.
Thanks for the help.
Cheers,
Mark Dalphin PhD
Director of Bioinformatics
mark.dalphin at pacificedge.co.nz
Ph: +64 3 479 4690
Cell: +64 21 156 7625
Skype: mark.dalphin.pel
________________________________
87 St David St, PO Box 56, Dunedin, New Zealand 9016 www.pacificedgedx.com
________________________________________
From: Mark Dalphin <mark.dalphin at pacificedge.co.nz>
Sent: Friday, 22 April 2016 9:19
To: peter dalgaard; Prof Brian Ripley
Subject: Re: [Rd] Fresh build from source of R-3.2.5 failing "make
check" under 64-bit Ubuntu
I have run it both ways; for the purposes of this email, it was easier
to capture both STDOUT and STDERR ( '>&') into one file and have
the
process run in the background (second '&'). That way I could more
readily capture the exact commands used and not have them scroll off the
top of the screen (make's output is quite large). The exact same error
occurs when I run the 'make' command as:
./configure PARAMETERS
make
make check
It is on an NFS file system; I'll look into building on a local file
system (/usr/local/) and see if I get the same errors.
Cheers,
Mark Dalphin
On 22/04/16 00:35, peter dalgaard wrote:> On 21 Apr 2016, at 14:11 , Prof Brian Ripley <ripley at
stats.ox.ac.uk> wrote:
>
>> On 21/04/2016 12:20, peter dalgaard wrote:
>>> Hmm. I don't think this has been seen by other Ubuntu users, or
other Linuxen for that matter.
>>>
>>> You have a gratuitous use of "&", but I don't
really see how that would give these symptoms, unless you accidentally run a
command twice, creating a race condition between two background processes.
>> He did say he used tcsh, for which >& is legal and far more
memorable than (goes to look it up to get the order right) >foo 2>&1
> I know (old tcsh user, before bash got traction[*], in fact). It was the
"&" at the end that I was talking about:
>
> make check >& make_check.out &
>
> This runs make in the background and redirects stdout/err to a file. As I
said, it is most likely harmless unless you accidentally get two processes
running at the same time, but I think I'd try runnning in the foreground,
just in case.
>
> -pd
>
> [*] That'll be before Linux; the first versions of Slackware used bash
(ash for boot disks). So... like 1991, 25 years back. Yikes!
>
>>> More likely it is file system related, like NFS directory caching
or maybe permissions, although the latter seems unlikely if one package does
install properly. You might want to recheck whether installed.packages(lib.loc
= "myLib", priority = "NA") gives different results a bit
later, and if not, check that the package descriptions are sane.
>>>
>>> -pd
>>>
>>> On 21 Apr 2016, at 00:50 , Mark Dalphin <mark.dalphin at
pacificedge.co.nz> wrote:
>>>
>>>> Hi,
>>>>
>>>> Let me justify building R from source. While I can obtain R
built for me
>>>> under Ubuntu, I tend to build it from scratch as we use a
NFS-shared
>>>> build as well as shared R libraries for all the bioinfo staff
at this
>>>> company. Only one build and one set of packages to ensure
uniformity
>>>> across all workstations.
>>>>
>>>> My problem is that despite using a pretty standard build
process, I am
>>>> failing at the "make check" step. I don't see
what I might have done
>>>> that is causing the failure.
>>>>
>>>> My platform is:
>>>>
>>>>> uname -a
>>>> Linux littlebourne 3.13.0-85-generic #129-Ubuntu SMP Thu Mar 17
20:50:15
>>>> UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
>>>>
>>>>> cat /etc/issue
>>>> Ubuntu 14.04.4 LTS \n \l
>>>>
>>>> I've downloaded the recent R-3.2.5 tar-gz file and gone
through the
>>>> standard build, though in a non-standard location. I keep
copies of the
>>>> tar-gzipped file in an "Archive" directory.
>>>>
>>>>> gunzip -c Archive/R-3.2.5.tar.gz | tar xf -
>>>>> cd R-3.2.5/
>>>>> ./configure --prefix=/opt/apps/x86_64/R/R-3.2.5 LIBnn=lib
>>>> [ ... much output ...]
>>>> R is now configured for x86_64-pc-linux-gnu
>>>>
>>>> Source directory: .
>>>> Installation directory: /opt/apps/x86_64/R/R-3.2.5
>>>>
>>>> C compiler: gcc -std=gnu99 -g -O2
>>>> Fortran 77 compiler: gfortran -g -O2
>>>>
>>>> C++ compiler: g++ -g -O2
>>>> C++ 11 compiler: g++ -std=c++11 -g -O2
>>>> Fortran 90/95 compiler: gfortran -g -O2
>>>> Obj-C compiler: gcc -g -O2 -fobjc-exceptions
>>>>
>>>> Interfaces supported: X11, tcltk
>>>> External libraries: readline, zlib, bzlib, lzma, PCRE
>>>> Additional capabilities: PNG, JPEG, TIFF, NLS, cairo
>>>> Options enabled: shared BLAS, R profiling
>>>>
>>>> Capabilities skipped: ICU
>>>> Options not enabled: memory profiling
>>>>
>>>> Recommended packages: yes
>>>>
>>>> # Note: I use tcsh, not bash
>>>>> make >& make.out &
>>>> # There are no obvious errors and the 'make' proceeds
finally through
>>>> some Java configuration
>>>> # and then exits with a zero status.
>>>>
>>>>> make check >& make_check.out &
>>>> [ ... much output captured ...]
>>>> make[3]: Entering directory
`/opt/apps/x86_64/R/src/R-3.2.5/tests'
>>>> running code in 'reg-tests-1a.R' ... OK
>>>> running code in 'reg-tests-1b.R' ... OK
>>>> running code in 'reg-tests-1c.R' ... OK
>>>> running code in 'reg-tests-2.R' ... OK
>>>> comparing 'reg-tests-2.Rout' to
'./reg-tests-2.Rout.save' ... OK
>>>> running code in 'reg-examples1.R' ... OK
>>>> running code in 'reg-examples2.R' ... OK
>>>> running code in 'reg-packages.R' ...make[3]: ***
[reg-packages.Rout] Error 1
>>>> make[3]: Leaving directory
`/opt/apps/x86_64/R/src/R-3.2.5/tests'
>>>> make[2]: *** [test-Reg] Error 2
>>>> make[2]: Leaving directory
`/opt/apps/x86_64/R/src/R-3.2.5/tests'
>>>> make[1]: *** [test-all-basics] Error 1
>>>> make[1]: Leaving directory
`/opt/apps/x86_64/R/src/R-3.2.5/tests'
>>>> make: *** [check] Error 2
>>>>
>>>>> cd tests
>>>>> tail -30 reg-packages.Rout.fail
>>>> *** installing help indices
>>>> ** building package indices
>>>> ** testing if installed package can be loaded
>>>> * DONE (pkgB)
>>>> Loading required package: pkgB
>>>> building package exNSS4 ...
>>>>
>>>> installing package exNSS4 using file exNSS4_1.1.tar.gz ...
>>>> * installing *source* package 'exNSS4' ...
>>>> ** R
>>>> ** preparing package for lazy loading
>>>> Creating a generic function for 'plot' from package
'graphics' in
>>>> package 'exNSS4'
>>>> ** help
>>>> No man pages found in package 'exNSS4'
>>>> *** installing help indices
>>>> ** building package indices
>>>> ** testing if installed package can be loaded
>>>> * DONE (exNSS4)
>>>> Loading required package: exNSS4
>>>>> (res <- installed.packages(lib.loc = "myLib",
priority = "NA"))
>>>> Package LibPath Version Priority Depends Imports
LinkingTo Suggests
>>>> myTst "myTst" "myLib" "1.0" NA
"methods" NA NA NA
>>>> Enhances License License_is_FOSS
>>>> myTst NA "What license is it under?" NA
>>>> License_restricts_use OS_type MD5sum NeedsCompilation
Built
>>>> myTst NA NA NA NA
"3.2.5"
>>>>> stopifnot(identical(res[,"Package"],
setNames(,sort(c(p.lis, "myTst")))),
>>>> + res[,"LibPath"] == "myLib")
>>>> Error: identical(res[, "Package"], setNames(,
sort(c(p.lis, "myTst"))))
>>>> is not TRUE
>>>> Execution halted
>>>>
>>>>> ls myLib
>>>> exNSS4 myTst pkgA pkgB
>>>>
>>>> So, it looks like the "installed.packages()" is not
correctly detecting
>>>> what is present in "myLib".
>>>>
>>>> Has anyone else seen this? Has anyone got any ideas about what
is going
>>>> wrong? My environment does not include R_LIBS, LD_LIBRARY, etc.
The PATH
>>>> does include an older version of R, 3.1.1.
>>>>
>>>> Regards,
>>>> Mark Dalphin
>>>>
>>>> ______________________________________________
>>>> R-devel at r-project.org mailing list
>>>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>
>> --
>> Brian D. Ripley, ripley at stats.ox.ac.uk
>> Emeritus Professor of Applied Statistics, University of Oxford
peter dalgaard
2016-Apr-22 07:11 UTC
[Rd] Fresh build from source of R-3.2.5 failing "make check" under 64-bit Ubuntu [SOLVED]
> On 22 Apr 2016, at 00:25 , Mark Dalphin <mark.dalphin at pacificedge.co.nz> wrote: > >> >> mount | grep /opt/ > myHost.science:/mnt/home/opt/apps on /opt/apps type nfs (rw,soft,bg,nfsvers=3,addr=XXX.XXX.XXX.XXX) > > Just the NFS, I guess. This is not good, but not an R-Devel problem.On a hunch: Check out what the "soft" mount option does. I seem to recall some complications related to immediacy of operations. -pd -- Peter Dalgaard, Professor, Center for Statistics, Copenhagen Business School Solbjerg Plads 3, 2000 Frederiksberg, Denmark Phone: (+45)38153501 Office: A 4.23 Email: pd.mes at cbs.dk Priv: PDalgd at gmail.com