Andreas Kersting
2021-Mar-26 07:40 UTC
[Rd] custom allocators, Valgrind and uninitialized memory
Hi, In my package bettermc, I use a custom allocator, which hands out already defined/initialized memory (mmap of a POSIX shared memory object). If my code is run in R which was configured --with-valgrind-instrumentation > 0, Valgrind will (correctly) complain about uninitialized memory being used, e.g. ==813836== Conditional jump or move depends on uninitialised value(s) ==813836== at 0x4F0A9D: getvar (svn/R-devel/src/main/eval.c:5171) ==813836== by 0x4D9B38: bcEval (svn/R-devel/src/main/eval.c:6867) ==813836== by 0x4F0077: Rf_eval (svn/R-devel/src/main/eval.c:727) ==813836== by 0x4F09AF: forcePromise (svn/R-devel/src/main/eval.c:555) ==813836== by 0x4F0C57: FORCE_PROMISE (svn/R-devel/src/main/eval.c:5136) ==813836== by 0x4F0C57: getvar (svn/R-devel/src/main/eval.c:5177) ==813836== by 0x4D9B38: bcEval (svn/R-devel/src/main/eval.c:6867) ==813836== by 0x4F0077: Rf_eval (svn/R-devel/src/main/eval.c:727) ==813836== by 0x4F1A8D: R_execClosure (svn/R-devel/src/main/eval.c:1897) ==813836== by 0x4F2783: Rf_applyClosure (svn/R-devel/src/main/eval.c:1823) ==813836== by 0x4DF61D: bcEval (svn/R-devel/src/main/eval.c:7083) ==813836== by 0x4F0077: Rf_eval (svn/R-devel/src/main/eval.c:727) ==813836== by 0x4F1A8D: R_execClosure (svn/R-devel/src/main/eval.c:1897) ==813836== Uninitialised value was created by a client request ==813836== at 0x52D5CF: Rf_allocVector3 (svn/R-devel/src/main/memory.c:2892) ==813836== by 0x16B415EA: allocate_from_shm (packages/tests-vg/bettermc/src/copy2shm.c:289) ==813836== by 0x49D123: R_doDotCall (svn/R-devel/src/main/dotcode.c:614) ==813836== by 0x4DA36D: bcEval (svn/R-devel/src/main/eval.c:7671) ==813836== by 0x4F0077: Rf_eval (svn/R-devel/src/main/eval.c:727) ==813836== by 0x4F1A8D: R_execClosure (svn/R-devel/src/main/eval.c:1897) ==813836== by 0x4F2783: Rf_applyClosure (svn/R-devel/src/main/eval.c:1823) ==813836== by 0x4F0243: Rf_eval (svn/R-devel/src/main/eval.c:850) ==813836== by 0x49B68F: do_External (svn/R-devel/src/main/dotcode.c:573) ==813836== by 0x4D3566: bcEval (svn/R-devel/src/main/eval.c:7115) ==813836== by 0x4F0077: Rf_eval (svn/R-devel/src/main/eval.c:727) ==813836== by 0x4F1A8D: R_execClosure (svn/R-devel/src/main/eval.c:1897) (allocate_from_shm() is my function calling allocVector3() with a custom allocator.) Valgrind is correct, because allocVector3() explicitly calls VALGRIND_MAKE_MEM_UNDEFINED() on the memory my custom allocator returns. - Sould allocVector3() call VALGRIND_MAKE_MEM_UNDEFINED() also if a custom allocator is used? For some custom allocators this is correct, for others not. - Or should the code using a custom allocator call VALGRIND_MAKE_MEM_DEFINED() on the DATAPTR() returned by allocVector3()? E.g. ... ret = PROTECT(allocVector3(asInteger(type), asReal(length), &allocator)); VALGRIND_MAKE_MEM_DEFINED(DATAPTR(ret), size); ... For the latter to work also on systems without Valgrind installed, I need to include both valgrind.h and memcheck.h in the src of my package and include these (rather than the system headers), correct? Should I best take these headers directly from R (src/include/vg)? Thanks! Regards, Andreas
Andreas Kersting
2021-Mar-26 14:10 UTC
[Rd] custom allocators, Valgrind and uninitialized memory
Another idea for the second option. Instead of including the Valgrind headers,
the following could be enough:
#if __has_include(<valgrind/memcheck.h>)
#include <valgrind/memcheck.h>
#else
#define VALGRIND_MAKE_MEM_DEFINED(_qzz_addr,_qzz_len) \
do { \
(_qzz_addr); \
(_qzz_len); \
} while (0)
#endif
I guess the packages are build on the same CRAN machine which also runs the
tests under Valgrind, i.e. valgrind/memcheck.h is available during build of the
package!?
Not sure though if Oracle Developer Studio on Solaris supports __has_include()
...
2021-03-26 08:40 GMT+01:00 "Andreas Kersting" <r-devel at
akersting.de>:> Hi,
>
> In my package bettermc, I use a custom allocator, which hands out already
defined/initialized memory (mmap of a POSIX shared memory object).
>
> If my code is run in R which was configured --with-valgrind-instrumentation
> 0, Valgrind will (correctly) complain about uninitialized memory being
used, e.g.
>
> ==813836== Conditional jump or move depends on uninitialised value(s)
> ==813836== at 0x4F0A9D: getvar (svn/R-devel/src/main/eval.c:5171)
> ==813836== by 0x4D9B38: bcEval (svn/R-devel/src/main/eval.c:6867)
> ==813836== by 0x4F0077: Rf_eval (svn/R-devel/src/main/eval.c:727)
> ==813836== by 0x4F09AF: forcePromise (svn/R-devel/src/main/eval.c:555)
> ==813836== by 0x4F0C57: FORCE_PROMISE (svn/R-devel/src/main/eval.c:5136)
> ==813836== by 0x4F0C57: getvar (svn/R-devel/src/main/eval.c:5177)
> ==813836== by 0x4D9B38: bcEval (svn/R-devel/src/main/eval.c:6867)
> ==813836== by 0x4F0077: Rf_eval (svn/R-devel/src/main/eval.c:727)
> ==813836== by 0x4F1A8D: R_execClosure (svn/R-devel/src/main/eval.c:1897)
> ==813836== by 0x4F2783: Rf_applyClosure
(svn/R-devel/src/main/eval.c:1823)
> ==813836== by 0x4DF61D: bcEval (svn/R-devel/src/main/eval.c:7083)
> ==813836== by 0x4F0077: Rf_eval (svn/R-devel/src/main/eval.c:727)
> ==813836== by 0x4F1A8D: R_execClosure (svn/R-devel/src/main/eval.c:1897)
> ==813836== Uninitialised value was created by a client request
> ==813836== at 0x52D5CF: Rf_allocVector3
(svn/R-devel/src/main/memory.c:2892)
> ==813836== by 0x16B415EA: allocate_from_shm
(packages/tests-vg/bettermc/src/copy2shm.c:289)
> ==813836== by 0x49D123: R_doDotCall (svn/R-devel/src/main/dotcode.c:614)
> ==813836== by 0x4DA36D: bcEval (svn/R-devel/src/main/eval.c:7671)
> ==813836== by 0x4F0077: Rf_eval (svn/R-devel/src/main/eval.c:727)
> ==813836== by 0x4F1A8D: R_execClosure (svn/R-devel/src/main/eval.c:1897)
> ==813836== by 0x4F2783: Rf_applyClosure
(svn/R-devel/src/main/eval.c:1823)
> ==813836== by 0x4F0243: Rf_eval (svn/R-devel/src/main/eval.c:850)
> ==813836== by 0x49B68F: do_External (svn/R-devel/src/main/dotcode.c:573)
> ==813836== by 0x4D3566: bcEval (svn/R-devel/src/main/eval.c:7115)
> ==813836== by 0x4F0077: Rf_eval (svn/R-devel/src/main/eval.c:727)
> ==813836== by 0x4F1A8D: R_execClosure (svn/R-devel/src/main/eval.c:1897)
>
> (allocate_from_shm() is my function calling allocVector3() with a custom
allocator.) Valgrind is correct, because allocVector3() explicitly calls
VALGRIND_MAKE_MEM_UNDEFINED() on the memory my custom allocator returns.
>
> - Sould allocVector3() call VALGRIND_MAKE_MEM_UNDEFINED() also if a custom
allocator is used? For some custom allocators this is correct, for others not.
>
> - Or should the code using a custom allocator call
VALGRIND_MAKE_MEM_DEFINED() on the DATAPTR() returned by allocVector3()? E.g.
>
> ...
> ret = PROTECT(allocVector3(asInteger(type), asReal(length),
&allocator));
> VALGRIND_MAKE_MEM_DEFINED(DATAPTR(ret), size);
> ...
>
> For the latter to work also on systems without Valgrind installed, I need
to include both valgrind.h and memcheck.h in the src of my package and include
these (rather than the system headers), correct? Should I best take these
headers directly from R (src/include/vg)?
>
> Thanks! Regards,
> Andreas
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>