R devel - Jan 2015 - Request for help with UBSAN and total absense of CRAN response

CRAN has a package of mine in upload limbo because it failed UBSAN.

I am not entirely ignorant on the topic of sanitizers and SAN / ASAN / UBSAN;
we created not one but two Docker containers with ASAN and USBAN:

   https://registry.hub.docker.com/u/rocker/r-devel-san/
   https://registry.hub.docker.com/u/rocker/r-devel-ubsan-clang/

as well as predecessors to them in earlier Docker repos.

Yet I fail to recreate the errors reported by CRAN:

  
http://www.stats.ox.ac.uk/pub/bdr/memtests/UBSAN-clang-trunk/RcppAnnoy/tests/runUnitTests.Rout
  
http://www.stats.ox.ac.uk/pub/bdr/memtests/UBSAN/RcppAnnoy/tests/runUnitTests.Rout

I asked politely (and twice) for help with the corresponding compiler
configuration(s).  But CRAN is of course way above communicating with mere
mortals such as yours truly.

So I have no recourse other than to spam all of you: if anybody here has a
working UBSAN setup which can replicate the issue seen in the (rather small)
RcppAnnoy package?

Erik (upstream for Annoy, CC'ed) and I would be most grateful.  We do not
like being held hostage on an error report we cannot replicate and for which
we do not receive any help (or even further communication) whatsoever.

Dirk
about to turn into yet another frustrated CRAN user

-- 
http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org

Hello, Dirk.

I have no experience with UBSAN etc., but as you mention not being
able to recreate the CRAN errors, it reminds me of the time I had
trouble getting my pacakge approved for CRAN as it passed all tets but
Win32, and I have a Win64 setup and could not test it. Have you tried
submitting the package to
<http://win-builder.r-project.org/upload.aspx> and seeing if the CRAN
errors get created there, or is the problem CRAN shows on platforms
other than Windows?

Avi

On Tue, Jan 13, 2015 at 10:30 AM, Dirk Eddelbuettel <edd at debian.org>
wrote:
>
> CRAN has a package of mine in upload limbo because it failed UBSAN.
>
> I am not entirely ignorant on the topic of sanitizers and SAN / ASAN /
UBSAN;
> we created not one but two Docker containers with ASAN and USBAN:
>
>    https://registry.hub.docker.com/u/rocker/r-devel-san/
>    https://registry.hub.docker.com/u/rocker/r-devel-ubsan-clang/
>
> as well as predecessors to them in earlier Docker repos.
>
> Yet I fail to recreate the errors reported by CRAN:
>
>   
http://www.stats.ox.ac.uk/pub/bdr/memtests/UBSAN-clang-trunk/RcppAnnoy/tests/runUnitTests.Rout
>   
http://www.stats.ox.ac.uk/pub/bdr/memtests/UBSAN/RcppAnnoy/tests/runUnitTests.Rout
>
> I asked politely (and twice) for help with the corresponding compiler
> configuration(s).  But CRAN is of course way above communicating with mere
> mortals such as yours truly.
>
> So I have no recourse other than to spam all of you: if anybody here has a
> working UBSAN setup which can replicate the issue seen in the (rather
small)
> RcppAnnoy package?
>
> Erik (upstream for Annoy, CC'ed) and I would be most grateful.  We do
not
> like being held hostage on an error report we cannot replicate and for
which
> we do not receive any help (or even further communication) whatsoever.
>
> Dirk
> about to turn into yet another frustrated CRAN user
>
> --
> http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

----- Original Message -----
> From: "Dirk Eddelbuettel" <edd at debian.org>
> To: r-devel at r-project.org
> Cc: "Erik Bernhardsson" <erik at malfunction.org>,
"Dirk Eddelbuettel" <edd at debian.org>
> Sent: Tuesday, January 13, 2015 7:30:47 AM
> Subject: [Rd] Request for help with UBSAN and total absense of CRAN
response
> 
> 
> CRAN has a package of mine in upload limbo because it failed UBSAN.
> 
> I am not entirely ignorant on the topic of sanitizers and SAN / ASAN
> / UBSAN;
> we created not one but two Docker containers with ASAN and USBAN:
> 
>    https://registry.hub.docker.com/u/rocker/r-devel-san/
>    https://registry.hub.docker.com/u/rocker/r-devel-ubsan-clang/
> 
> as well as predecessors to them in earlier Docker repos.
> 
> Yet I fail to recreate the errors reported by CRAN:
> 
>   
http://www.stats.ox.ac.uk/pub/bdr/memtests/UBSAN-clang-trunk/RcppAnnoy/tests/runUnitTests.Rout
>   
http://www.stats.ox.ac.uk/pub/bdr/memtests/UBSAN/RcppAnnoy/tests/runUnitTests.Rout
> 
> I asked politely (and twice) for help with the corresponding compiler
> configuration(s).  But CRAN is of course way above communicating with
> mere
> mortals such as yours truly.
> 
> So I have no recourse other than to spam all of you: if anybody here
> has a
> working UBSAN setup which can replicate the issue seen in the (rather
> small)
> RcppAnnoy package?
> 
Where should the package source be downloaded from? I see it in CRAN (but
presumably the latest version that causes the issue is not yet downloadable) and
in github.

Dan
> Erik (upstream for Annoy, CC'ed) and I would be most grateful.  We do
> not
> like being held hostage on an error report we cannot replicate and
> for which
> we do not receive any help (or even further communication)
> whatsoever.
> 
> Dirk
> about to turn into yet another frustrated CRAN user
> 
> --
> http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org
> 
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

On 13 January 2015 at 08:21, Dan Tenenbaum wrote:
| Where should the package source be downloaded from? I see it in CRAN (but
presumably the latest version that causes the issue is not yet downloadable) and
in github.

The "presumable" assumption is incorrect AFAIK. 

The error should presumably came up in both versions as annoylib.h did not
change there.  Feel free to prove me wrong :) and just use whatever is easiest.

Dirk

-- 
http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org

Dirk,

The vagrant setup I use to test my packages with UBSAN also seems to  
replicate the error reported by CRAN (together with some other  
warnings). I have attached the files (I hope they get through the  
filters). I suppose you know what to do with them.

Jan





Dirk Eddelbuettel <edd at debian.org> schreef:
> CRAN has a package of mine in upload limbo because it failed UBSAN.
>
> I am not entirely ignorant on the topic of sanitizers and SAN / ASAN /
UBSAN;
> we created not one but two Docker containers with ASAN and USBAN:
>
>    https://registry.hub.docker.com/u/rocker/r-devel-san/
>    https://registry.hub.docker.com/u/rocker/r-devel-ubsan-clang/
>
> as well as predecessors to them in earlier Docker repos.
>
> Yet I fail to recreate the errors reported by CRAN:
>
>     
>
http://www.stats.ox.ac.uk/pub/bdr/memtests/UBSAN-clang-trunk/RcppAnnoy/tests/runUnitTests.Rout
>     
>
http://www.stats.ox.ac.uk/pub/bdr/memtests/UBSAN/RcppAnnoy/tests/runUnitTests.Rout
>
> I asked politely (and twice) for help with the corresponding compiler
> configuration(s).  But CRAN is of course way above communicating with mere
> mortals such as yours truly.
>
> So I have no recourse other than to spam all of you: if anybody here has a
> working UBSAN setup which can replicate the issue seen in the (rather
small)
> RcppAnnoy package?
>
> Erik (upstream for Annoy, CC'ed) and I would be most grateful.  We do
not
> like being held hostage on an error report we cannot replicate and for
which
> we do not receive any help (or even further communication) whatsoever.
>
> Dirk
> about to turn into yet another frustrated CRAN user
>
> --
> http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

... and they didn't make it through. I put the files in a gist:

https://gist.github.com/djvanderlaan/1e9beb75d2d595824efc

Jan



On 16-01-15 15:21, Jan van der Laan wrote:
> Dirk,
>
> The vagrant setup I use to test my packages with UBSAN also seems to
> replicate the error reported by CRAN (together with some other
> warnings). I have attached the files (I hope they get through the
> filters). I suppose you know what to do with them.
>
> Jan
>
>
>
>
>
> Dirk Eddelbuettel <edd at debian.org> schreef:
>
>> CRAN has a package of mine in upload limbo because it failed UBSAN.
>>
>> I am not entirely ignorant on the topic of sanitizers and SAN / ASAN /
>> UBSAN;
>> we created not one but two Docker containers with ASAN and USBAN:
>>
>>    https://registry.hub.docker.com/u/rocker/r-devel-san/
>>    https://registry.hub.docker.com/u/rocker/r-devel-ubsan-clang/
>>
>> as well as predecessors to them in earlier Docker repos.
>>
>> Yet I fail to recreate the errors reported by CRAN:
>>
>>
>>
http://www.stats.ox.ac.uk/pub/bdr/memtests/UBSAN-clang-trunk/RcppAnnoy/tests/runUnitTests.Rout
>>
>>
>>
http://www.stats.ox.ac.uk/pub/bdr/memtests/UBSAN/RcppAnnoy/tests/runUnitTests.Rout
>>
>>
>> I asked politely (and twice) for help with the corresponding compiler
>> configuration(s).  But CRAN is of course way above communicating with
>> mere
>> mortals such as yours truly.
>>
>> So I have no recourse other than to spam all of you: if anybody here
>> has a
>> working UBSAN setup which can replicate the issue seen in the (rather
>> small)
>> RcppAnnoy package?
>>
>> Erik (upstream for Annoy, CC'ed) and I would be most grateful.  We
do not
>> like being held hostage on an error report we cannot replicate and for
>> which
>> we do not receive any help (or even further communication) whatsoever.
>>
>> Dirk
>> about to turn into yet another frustrated CRAN user
>>
>> --
>> http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org
>>
>> ______________________________________________
>> R-devel at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

I would like to express my deepest gratitude to the CRAN Maintainers for
keeping up with their perfect record of never responding to questions from
mere mortals like myself. The ensuing utter and complete silence allowed me
to work with a much higher concentration level.

More seriously, I owe a big thank you to Martyn Plummer and Jan van der Laan
for replying here (and in off-list follow-ups), and to Jeff Horner who
offered help via Twitter and off-list emails.  Thanks to their help, I did
get to the bottom of this, and it seems that part of (either mine only, or a
more general) confusion was that I wanted the test to actually _abort_ on
error.  Which, as I found after too many failed tries, requires also setting
the -fno-sanitize-recover option as well.

Long story short, we now have a working "appliance" to test this via
Docker.

This is building on some of the examples I showed at the end of my useR! 2014
presentation, the work Carl and I have been doing in the Rocker.org repo for
Docker containers for R, some experimentation, and an only-at-GitHub-yet new
script for littler.  I hope to blog about this in some more detail, but if
you have Docker set-up, pull the rocker/r-devel-ubsan-clang container.  

Then:

i)  to replicate the reported error:

    docker run --rm -ti -v $(pwd):/mnt rocker/r-devel-ubsan-clang \
           check.r --setwd /mnt --install-deps RcppAnnoy_0.0.5.tar.gz 

    which runs the container with 
      - post-run cleanup (--rm), 
      - terminal and interactive mode (-ti), 
      - the current directory mounted as /mnt in the container (-v ...),
      - invoking the check.r script (from littler as on GitHub, in the path in
the
        container) with options to install package dependencies
      - the known-bad tarball

ii) to see a fix not triggering it

    docker run --rm -ti -v $(pwd):/mnt rocker/r-devel-ubsan-clang \
           check.r --setwd /mnt --install-deps RcppAnnoy_0.0.5.1.tar.gz 

    as above but using an updated tarball not showing the error.

This is generic: you can plug any of your R package tarballs in there
and it will run the test.  [ This assumes dependencies can be satisfied via
install.packages(); need to add a hook for apt-get as needed etc pp ]

The UBSAN config I settled on for now is close to what Martyn had sent me (in
off-list follow-up):

 CC="clang-3.5 -fsanitize=undefined \
      -fno-sanitize=float-divide-by-zero,vptr,function
-fno-sanitize-recover"
 CXX="clang++-3.5 -fsanitize=undefined \
      -fno-sanitize=float-divide-by-zero,vptr,function
-fno-sanitize-recover"
 CXX1X="clang++-3.5 -fsanitize=undefined \
      -fno-sanitize=float-divide-by-zero,vptr,function
-fno-sanitize-recover"

We may want to turn on other options; Jeff Horner sent me his config which
has a lot more goodies enabled.  

I'd welcome follow-up by anyone interested in working on this, either by
(off-list) email or directly over at the GitHub repo for this container:
   https://github.com/rocker-org/r-devel-san-clang

Thanks,  Dirk

-- 
http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org