R devel - Jan 2023 - rhub vs. CRAN fedora-*-devel, using armadillo & slapack

Thank you for your answer.
In facts, 10 threads are asked by armadillo for some LinAlg, which backs to two
threads as warned. But I cannot imagine this costs so much time just for that...

A deeper analysis of time spent seems to point that a large time was mainly
spent on testthat and Rcpp dependencies compilation... But other recent packages
depending on these also are not spending so much time.

CRAN team, as I failed to reproduce the issue with rhub dockers, 
- is there any reason that could explain that fedora-*-devel is so slow for this
package or compilation of Rcpp/testthat ?
- is our slapack a possible source of... anything ?
- are we the only package which embeds "standard armadillo" and tries
to deal with simple precision lapack issues on fedora ?
- is there any chance that I can get a deeper log of what happened ?

Best regards,

Yann

-----Message d'origine-----
De?: Serguei Sokol <sokol at insa-toulouse.fr> 
Envoy??: mardi 10 janvier 2023 11:41
??: RICHET Yann <yann.richet at irsn.fr>; R-devel at r-project.org
Cc?: Pascal Hav? <pascal at haveneer.com>
Objet?: Re: [Rd] rhub vs. CRAN fedora-*-devel, using armadillo & slapack

Le 10/01/2023 ? 11:37, Serguei Sokol a ?crit?:
> Le 10/01/2023 ? 10:44, RICHET Yann a ?crit?:
>> Dear R-devel people,
>>
>> We are working to submit a package which is mainly a binding over a
>> C++ lib (https://github.com/libKriging) using armadillo.
>> It is _not_ a standard RcppArmadillo package, because we also had to 
>> provide a python binding... so there is a huge layer of cmake & 
>> scripting to make it work with a standard armadillo (but using same 
>> version that RcppArmadillo).
>> It seems now working with almost all CRAN targets, but a problem 
>> remains with fedora (clang & gcc) devel.
>>
>> Our issue is that the rhub docker is not well sync with the CRAN 
>> fedora-clang-devel config:
>> - failing on CRAN (without clear reason): 
>> https://www.r-project.org/nosvn/R.check/r-devel-linux-x86_64-fedora-c
>> lang/rlibkriging-00check.html
> I see? 2 candidates for? reasons of failing on CRAN:
> ?1. test time is 30 min while 15 min are allowed;
> ?2. your code try to get 30 threads
Oops, it was 10 threads not 30 but it does not change the nature of a possible
issue.
> while CRAN limits them to 2;
>
> Try to make your tests shorter ( < 15 min) on 2 threads. May be it 
> will help.
>
> Best,
> Serguei.
>
>> - passing on rhub: 
>> https://builder.r-hub.io/status/rlibkriging_0.7-3.tar.gz-20f7dc756272
>> 6497af7c678ab41f4dea
>>
>> So we cannot investigate and fix the problem.
>>
>> Note that we did quite strange things with the fedora platforms: 
>> include explicitely slapack to provide simple precision for our
>> (vanilla) armadillo...
>>
>> Do you have any idea, or even known problem in mind, that could be 
>> related to this ?
>>
>> Best regards,
>>
>> --
>> Dr. Yann Richet
>> Institute for Radiological Protection and Nuclear Safety 
>> (https://www.irsn.fr),
>> ?? Department of Characterization of Natural Unexpected Events and 
>> Sites Office?: +33 1 58 35 88 84
>>
>> ______________________________________________
>> R-devel at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>
>

--
Serguei Sokol
Ingenieur de recherche INRAE

Cellule Math?matiques
TBI, INSA/INRAE UMR 792, INSA/CNRS UMR 5504
135 Avenue de Rangueil
31077 Toulouse Cedex 04

tel: +33 5 61 55 98 49
email: sokol at insa-toulouse.fr
http://www.toulouse-biotechnology-institute.fr/en/technology_platforms/mathematics-cell.html

Looks like there is a kind of misunderstanding...

Le 10/01/2023 ? 17:27, RICHET Yann a ?crit?:
> Thank you for your answer.
> In facts, 10 threads are asked by armadillo for some LinAlg, which backs to
two threads as warned. But I cannot imagine this costs so much time just for
that...
Excessive thread number is a problem per se. I did not say that it was 
responsible for excessive test time.
It's up to you to configure armadillo to use no more then 2 threads when 
run on CRAN. May be, setting environment variable OPENBLAS_NUM_THREADS=2 
could help.
>
> A deeper analysis of time spent seems to point that a large time was mainly
spent on testthat and Rcpp dependencies compilation...
Normally, compilation time is not accounted in the quota of 15 min 
dedicated to tests.
I would just focus on reducing this time, e.g. by running tests on 
smaller cases or skipping time-consuming tests conditionally when on CRAN.

Serguei.
>   But other recent packages depending on these also are not spending so
much time.
>
> CRAN team, as I failed to reproduce the issue with rhub dockers,
> - is there any reason that could explain that fedora-*-devel is so slow for
this package or compilation of Rcpp/testthat ?
> - is our slapack a possible source of... anything ?
> - are we the only package which embeds "standard armadillo" and
tries to deal with simple precision lapack issues on fedora ?
> - is there any chance that I can get a deeper log of what happened ?
>
> Best regards,
>
> Yann
>
> -----Message d'origine-----
> De?: Serguei Sokol<sokol at insa-toulouse.fr>  
> Envoy??: mardi 10 janvier 2023 11:41
> ??: RICHET Yann<yann.richet at irsn.fr>;R-devel at r-project.org
> Cc?: Pascal Hav?<pascal at haveneer.com>
> Objet?: Re: [Rd] rhub vs. CRAN fedora-*-devel, using armadillo &
slapack
>
> Le 10/01/2023 ? 11:37, Serguei Sokol a ?crit?:
>> Le 10/01/2023 ? 10:44, RICHET Yann a ?crit?:
>>> Dear R-devel people,
>>>
>>> We are working to submit a package which is mainly a binding over a
>>> C++ lib (https://github.com/libKriging) using armadillo.
>>> It is _not_ a standard RcppArmadillo package, because we also had
to
>>> provide a python binding... so there is a huge layer of cmake &
>>> scripting to make it work with a standard armadillo (but using same
>>> version that RcppArmadillo).
>>> It seems now working with almost all CRAN targets, but a problem
>>> remains with fedora (clang & gcc) devel.
>>>
>>> Our issue is that the rhub docker is not well sync with the CRAN
>>> fedora-clang-devel config:
>>> - failing on CRAN (without clear reason):
>>>
https://www.r-project.org/nosvn/R.check/r-devel-linux-x86_64-fedora-c
>>> lang/rlibkriging-00check.html
>> I see? 2 candidates for? reasons of failing on CRAN:
>>  ?1. test time is 30 min while 15 min are allowed;
>>  ?2. your code try to get 30 threads
> Oops, it was 10 threads not 30 but it does not change the nature of a
possible issue.
>
>> while CRAN limits them to 2;
>>
>> Try to make your tests shorter ( < 15 min) on 2 threads. May be it
>> will help.
>>
>> Best,
>> Serguei.
>>
>>> - passing on rhub:
>>>
https://builder.r-hub.io/status/rlibkriging_0.7-3.tar.gz-20f7dc756272
>>> 6497af7c678ab41f4dea
>>>
>>> So we cannot investigate and fix the problem.
>>>
>>> Note that we did quite strange things with the fedora platforms:
>>> include explicitely slapack to provide simple precision for our
>>> (vanilla) armadillo...
>>>
>>> Do you have any idea, or even known problem in mind, that could be
>>> related to this ?
>>>
>>> Best regards,
>>>
>>> --
>>> Dr. Yann Richet
>>> Institute for Radiological Protection and Nuclear Safety
>>> (https://www.irsn.fr),
>>>  ?? Department of Characterization of Natural Unexpected Events and
>>> Sites Office?: +33 1 58 35 88 84
>>>
>>> ______________________________________________
>>> R-devel at r-project.org  mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>
	[[alternative HTML version deleted]]

On Tue, 10 Jan 2023 16:27:53 +0000
RICHET Yann <yann.richet at irsn.fr> wrote:
> In facts, 10 threads are asked by armadillo for some LinAlg, which
> backs to two threads as warned. 
I think you're right about your tests de-facto using two threads, but
it might be a good idea to _default_ to up to two threads in tests and
examples. This is especially valuable for third-party developers who
have to mass-test packages (one of which could be rlibkriging) in
parallel.
> - is there any reason that could explain that fedora-*-devel is so
> slow for this package or compilation of Rcpp/testthat ?
Compilation time is definitely not the reason. Something in tests/*
actually runs for 30 minutes by itself.
> - is there any chance that I can get a deeper log of what happened ?
If you split your tests into separate files under tests/*.R instead of
using a single tests/testthat.R calling the rest of the tests, R will
be able to show you the individual test file that hung and maybe the
line where it happened. (Also, you'll get per-file timing.) But that
is potentially a huge investment: you may have to rewrite the tests to
work outside the testthat harness, and you'd also have to prepare
another CRAN submission just to have those tests run. It's also against
CRAN policy to knowingly submit a package with unfixed ERRORs.

(Currently, R can only tell you that the tests hung in the
test_check('rlibkriging') call in the tests/testthat.R, which isn't
precise enough.)

These lines look a bit scary:

tests/testthat/bench-KrigingFit.R:

pack=list.files(file.path("bindings","R"),pattern =
".tar.gz",full.names = T) install.packages(pack,repos=NULL)

tests/testthat/notest-LinearRegression.R:

install.packages(pkgs="rlibkriging_0.7-2.tgz",
type="source", repos=NULL)
library(rlibkriging)

I don't think that install.packages does anything besides raising a
warning and letting the test continue (there doesn't seem to be any
.tar.gz files inside the package on CRAN, so installation fails), but
it's probably not a good idea to install packages during testing [*].

This is also not quite right but harmless, because rlibkriging is
already loaded:

library(rlibkriging, lib.loc="bindings/R/Rlibs")

I'm afraid I don't know how to reproduce the timeouts you're seeing
on
the CRAN Fedora machine.

-- 
Best regards,
Ivan

[*] I once wrote a test that launched multiple child R processes,
created a temporary library in each of them and installed the package
there. The test was designed to only run on the developer's machine in
order to test file format compatibility between different versions of
R. I ended up moving the test away from tests/*.R in order to make sure
it won't run by accident.