R devel - Mar 2021 - custom allocators, Valgrind and uninitialized memory

Hi Dirk,

Sure, let me try to explain:

CRAN ran the tests of my package using R which was configured
--with-valgrind-instrumentation > 0. Valgrind reported many errors related to
the use of supposedly uninitialized memory and the CRAN team asked me to tackle
these.

These errors are false positives, because I pass a custom allocator to
allocVector3() which hands out memory which is already initialized. However,
this memory is explicitly marked for Valgrind as uninitialized by
allocVector3(), and I do not initialize it subsequently, so Valgrind complains.

Now I am asking if it is correct that allocVector3() marks memory as
uninitialized/undefined, even if it comes from a custom allocator. This is
because allocVector3() cannot know if the memory might already by initialized.

If this is the intended behavior of allocVector3(), then I am looking for the
proper way to get rid of these false Valgrind errors. So to be able to more
easily spot the true ones ...

Which section of _Writing R Extensions_ do you have in mind? I cannot find
anything on custom allocators there, but maybe I am using the wrong search
terms. No, these object are returned to R and I am not aware that this is a
problem / not allowed.

Regards, Andreas

2021-03-26 19:51 GMT+01:00 "Dirk Eddelbuettel" <edd at
debian.org>:
> 
> Andreas,
> 
> Can you briefly describe what it is you are trying to do?
> 
> In general, no R package would use valgrind directly; it is an optional
> debugger. Also note _Writing R Extensions_ has a few things to say about
how
> memory destined for R object can and cannot be allocated -- I presume your
> custom allocator is only for objects you managed and do not return to R?
> 
> Best, Dirk
> 
> -- 
> https://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org
>

Andreas,

Thanks for the clarification.

On 26 March 2021 at 20:48, Andreas Kersting wrote:
| Sure, let me try to explain:
| 
| CRAN ran the tests of my package using R which was configured
--with-valgrind-instrumentation > 0. Valgrind reported many errors related to
the use of supposedly uninitialized memory and the CRAN team asked me to tackle
these.
| 
| These errors are false positives, because I pass a custom allocator to
allocVector3() which hands out memory which is already initialized. However,
this memory is explicitly marked for Valgrind as uninitialized by
allocVector3(), and I do not initialize it subsequently, so Valgrind complains.
| 
| Now I am asking if it is correct that allocVector3() marks memory as
uninitialized/undefined, even if it comes from a custom allocator. This is
because allocVector3() cannot know if the memory might already by initialized.
| 
| If this is the intended behavior of allocVector3(), then I am looking for the
proper way to get rid of these false Valgrind errors. So to be able to more
easily spot the true ones ...
| 
| Which section of _Writing R Extensions_ do you have in mind? I cannot find
anything on custom allocators there, but maybe I am using the wrong search
terms. No, these object are returned to R and I am not aware that this is a
problem / not allowed.

I had the simpler section 6.1. in Writing R Extensions in mind, but you are
indeed using the allocVector3() interface from the public header in
R_ext/Rallocators.h -- but I only see scant mention of allocVector3 in R
Internals.  So I will have to pass. A search of CRAN via the Github mirror
[1] also reveals little ... but includes a hit from your package.

Dirk

[1] https://github.com/search?q=org%3Acran+allocVector3&type=code

-- 
https://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org

Hi Andreas,

On 3/26/21 8:48 PM, Andreas Kersting wrote:
> Hi Dirk,  > > Sure, let me try to explain: > > CRAN ran the
tests of my package
using R which was configured > --with-valgrind-instrumentation > 0. 
Valgrind reported many errors > related to the use of supposedly 
uninitialized memory and the CRAN > team asked me to tackle these. > > 
These errors are false positives, because I pass a custom allocator > to 
allocVector3() which hands out memory which is already > initialized. 
However, this memory is explicitly marked for Valgrind > as 
uninitialized by allocVector3(), and I do not initialize it > 
subsequently, so Valgrind complains. > > Now I am asking if it is 
correct that allocVector3() marks memory as > uninitialized/undefined, 
even if it comes from a custom allocator. > This is because 
allocVector3() cannot know if the memory might > already by initialized.
I think the semantics of allocVector/allocVector3 should be the same 
regardless of whether custom allocators are used. The semantics of 
allocVector is to provide uninitialized memory (non-pointer types, 
Writing R Extensions 5.9.2). Therefore, it is the caller who needs to 
take care of initialization. This is also the semantics of "malloc"
and
Rallocators.h says "custom_alloc_t mem_alloc; /* malloc equivalent
*/".

So I think that your code using your custom allocator needs to 
initialize allocated memory to be correct. If your allocator initializes 
the memory, that is fine, but unnecessary.

So technically speaking, the valgrind reports are not false alarms. I 
think your call sites should initialize.

Best
Tomas



	[[alternative HTML version deleted]]