R help - Apr 2015 - some general advice sought on the use of gctorture()

Dear all

I have bumped into the dreaded 'segfault' error type when running some
C++ code using .Call().

I have already undertaken several attempts to debug the C++ code with gdb(), but
until now I have been unable to pinpoint the origin of the problem. There are
two elements that I think are puzzling (a) this .Call() has worked fine for
about three years, for a variety of data (b)  the actual crash occurs at random
points during the execution of the function (well, random from a human eye's
point of view).
>From what I understand in the "R extensions" manual, the actual
problem may have been around for a while before the actual call to the C++ code.
As recommended in the manual, I am now using  gctorture() to try to pinpoint the
origins of the problem. I can, alas, only confirm that gctorture() has an
enormous impact on execution time, even for operations that are normally
executed within the blink of an eye. From what I have seen until now, executing
all the R code before the crash with gctorture(TRUE) could take months.
I suppose then that the best way to proceed would be to proceed backward from
the point where the crash occurs when gctorture(FALSE).

I have tried to find some concrete examples of good practices in the use of
gctorture() to identify memory problems in R, but most of what I have found on
the web is simply a copy of the help page. Does anybody know more concrete and
elaborated examples that could give an indication on how to best proceed
further?





Laurent Franckx, PhD
Senior researcher sustainable mobility
VITO NV | Boeretang 200 | 2400 Mol
Tel. ++ 32 14 33 58 22| mob. +32 479 25 59 07 | Skype: laurent.franckx |
laurent.franckx at vito.be | Twitter @LaurentFranckx




VITO Disclaimer: http://www.vito.be/e-maildisclaimer

On 04/24/2015 06:49 AM, Franckx Laurent wrote:
> Dear all
>
> I have bumped into the dreaded 'segfault' error type when running
some C++
> code using .Call().
segfaults often involve invalid memory access at the C level that are best 
discovered via valgrind or similar rather than gctorture. A good way to spot 
these is to

(a) come up with a _minimal_ reproducible script test.R that takes just a few 
seconds to run and that tickles, at least some times, the segfault

(b) make sure that your package is compiled without optimizations and with 
debugging symbols, e.g., in  ~/.R/Makevars add the lines

   CFLAGS="-ggdb -O0"
   CXXFLAGS="-ggdb -O0"

(c) run the code under 'valgrind'

   R -d valgrind -f test.r

Look especially for 'invalid read' or 'invalid write' messages,
and isolate
_your_ code in the callback that the message produces.

There is a 'worked example' at

   http://bioconductor.org/developers/how-to/c-debugging/#case-study

Of course this might lead to nothing, and then you'll be back to your
original
question about using gctorture or other strategies.

Martin Morgan
>
> I have already undertaken several attempts to debug the C++ code with
gdb(),
> but until now I have been unable to pinpoint the origin of the problem.
There
> are two elements that I think are puzzling (a) this .Call() has worked fine
> for about three years, for a variety of data (b)  the actual crash occurs
at
> random points during the execution of the function (well, random from a
human
> eye's point of view).
>
>> From what I understand in the "R extensions" manual, the
actual problem may
>> have been around for a while before the actual call to the C++ code. As
>> recommended in the manual, I am now using  gctorture() to try to
pinpoint
>> the origins of the problem. I can, alas, only confirm that gctorture()
has
>> an enormous impact on execution time, even for operations that are
normally
>> executed within the blink of an eye. From what I have seen until now,
>> executing all the R code before the crash with gctorture(TRUE) could
take
>> months.
>
> I suppose then that the best way to proceed would be to proceed backward
from
> the point where the crash occurs when gctorture(FALSE).
>
> I have tried to find some concrete examples of good practices in the use of
> gctorture() to identify memory problems in R, but most of what I have found
> on the web is simply a copy of the help page. Does anybody know more
concrete
> and elaborated examples that could give an indication on how to best
proceed
> further?
>
>
>
>
>
> Laurent Franckx, PhD Senior researcher sustainable mobility VITO NV |
> Boeretang 200 | 2400 Mol Tel. ++ 32 14 33 58 22| mob. +32 479 25 59 07 |
> Skype: laurent.franckx | laurent.franckx at vito.be | Twitter
@LaurentFranckx
>
>
>
>
> VITO Disclaimer: http://www.vito.be/e-maildisclaimer
>
> ______________________________________________ R-help at r-project.org
mailing
> list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting
guide
> http://www.R-project.org/posting-guide.html and provide commented, minimal,
> self-contained, reproducible code.
>

-- 
Computational Biology / Fred Hutchinson Cancer Research Center
1100 Fairview Ave. N.
PO Box 19024 Seattle, WA 98109

Location: Arnold Building M1 B861
Phone: (206) 667-2793

This is very off-topic here. My suggestion would be to do as the Posting Guide
says and ask this on R-devel, or perhaps even a gdb forum. From what little I
know, valgrind might help also.
---------------------------------------------------------------------------
Jeff Newmiller                        The     .....       .....  Go Live...
DCN:<jdnewmil at dcn.davis.ca.us>        Basics: ##.#.       ##.#.  Live
Go...
                                      Live:   OO#.. Dead: OO#..  Playing
Research Engineer (Solar/Batteries            O.O#.       #.O#.  with
/Software/Embedded Controllers)               .OO#.       .OO#.  rocks...1k
--------------------------------------------------------------------------- 
Sent from my phone. Please excuse my brevity.

On April 24, 2015 6:49:31 AM PDT, Franckx Laurent <laurent.franckx at
vito.be> wrote:
>Dear all
>
>I have bumped into the dreaded 'segfault' error type when running
some
>C++ code using .Call().
>
>I have already undertaken several attempts to debug the C++ code with
>gdb(), but until now I have been unable to pinpoint the origin of the
>problem. There are two elements that I think are puzzling (a) this
>.Call() has worked fine for about three years, for a variety of data
>(b)  the actual crash occurs at random points during the execution of
>the function (well, random from a human eye's point of view).
>
>>From what I understand in the "R extensions" manual, the
actual
>problem may have been around for a while before the actual call to the
>C++ code. As recommended in the manual, I am now using  gctorture() to
>try to pinpoint the origins of the problem. I can, alas, only confirm
>that gctorture() has an enormous impact on execution time, even for
>operations that are normally executed within the blink of an eye. From
>what I have seen until now, executing all the R code before the crash
>with gctorture(TRUE) could take months.
>
>I suppose then that the best way to proceed would be to proceed
>backward from the point where the crash occurs when gctorture(FALSE).
>
>I have tried to find some concrete examples of good practices in the
>use of gctorture() to identify memory problems in R, but most of what I
>have found on the web is simply a copy of the help page. Does anybody
>know more concrete and elaborated examples that could give an
>indication on how to best proceed further?
>
>
>
>
>
>Laurent Franckx, PhD
>Senior researcher sustainable mobility
>VITO NV | Boeretang 200 | 2400 Mol
>Tel. ++ 32 14 33 58 22| mob. +32 479 25 59 07 | Skype: laurent.franckx
>| laurent.franckx at vito.be | Twitter @LaurentFranckx
>
>
>
>
>VITO Disclaimer: http://www.vito.be/e-maildisclaimer
>
>______________________________________________
>R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
>https://stat.ethz.ch/mailman/listinfo/r-help
>PLEASE do read the posting guide
>http://www.R-project.org/posting-guide.html
>and provide commented, minimal, self-contained, reproducible code.