R devel - Oct 2007 - append/concatenate an element to a list in C-language

dear people,

i need to code a function in C working in R and receives two R SEXP
objects as parameters, where one is a list and another is a vector of
integers:

void f(SEXP list, SEXP vector) {

  ...

  return list;
}


and it should return the given list with the integer vector concatenated
at the end (as the last element of the list). the list can be really big
so i would not like to create a new list from scratch and copy all the
elements, including the additional one. i'm also interested in knowing
how should i properly handle protections of the SEXP objects when doing
this.

i've been looking at the R source code for everything that has to do
with CAR, CDR, CONS, and even found functions with promising names like
listAppend or GrowList but i have not been able to figure this out nor i
haven't been able to find any reference on how to do this by googling
all around, so any help will be very much appreciated.


thanks a lot!!!

robert.

Hi.

I believe it is virtually impossible, maybe even not virtually: if you
manage to do so somehow, please report it as a bug because such things
must be impossible as they break the integrity of R and data.

Forget about changing size in place: it is C and 'realloc' would be
about the only way to do it, which would not be smart even if you can
find a pointer to what you want to change. However, consider you want to
copy (after all it is a list, which is an array of pointers and you
might think you only need to copy pointers, which is cheap even for a
huge list). Consider a simple example:
you allocVector(VECSXP,n+1); then either memcpy n elements (if you
manage to get a pointer say with DATAPTR, and R-API is smart not to
allow you to get it that simple) or you simply SET_VECTOR_ELT(new, i,
VECTOR_ELT(old, i)) and then add the last one. You know what would be
wrong about this idea? You cannot be sure that pointers to elements of
the old list are not modified/deleted elsewhere, thus affecting your new
list! Even if you overwrite the list in place, providing such software
to users is error prone as users do not know about your ideas of keeping
everything tidy.

The only right way to replicate (if you want to do it in C) is copy and
copy duplicating each element of the list, i.e.:

SEXP f(SEXP old, SEXP v) {
SEXP new; int i,nprotect=0;
PROTECT(new=allocVector(VECSXP,n+1)); nprotect++;
for(i=0;i<n;i++)
  SET_VECTOR_ELT(new, i, Rf_duplicate(VECTOR_ELT(old, i)));
SET_VECTOR_ELT(new, n, v);
/* set names here if you need */
UNPROTECT(1);
return new;

What you could do, if you really need such functionality. Provide R with
external pointer to a data structure that you maintain in your C (I
would suggest C++) code and use C++ STL for dynamic updates (either
lists or vectors, whatever suits better). But the overhead of writing
the interface will take more than copying a list of any size!

Finally: void f(SEXP list, SEXP vector); is a malformed definition which
will lead to Segmentation Fault. Please read "Writing R extensions"
first.

Best,
Oleg

-  
Dr Oleg Sklyar * EMBL-EBI, Cambridge CB10 1SD, UK * +441223494466


On Thu, 2007-10-18 at 19:41 +0200, Robert Castelo wrote:
> dear people,
> 
> i need to code a function in C working in R and receives two R SEXP
> objects as parameters, where one is a list and another is a vector of
> integers:
> 
> void f(SEXP list, SEXP vector) {
> 
>   ...
> 
>   return list;
> }
> 
> 
> and it should return the given list with the integer vector concatenated
> at the end (as the last element of the list). the list can be really big
> so i would not like to create a new list from scratch and copy all the
> elements, including the additional one. i'm also interested in knowing
> how should i properly handle protections of the SEXP objects when doing
> this.
> 
> i've been looking at the R source code for everything that has to do
> with CAR, CDR, CONS, and even found functions with promising names like
> listAppend or GrowList but i have not been able to figure this out nor i
> haven't been able to find any reference on how to do this by googling
> all around, so any help will be very much appreciated.
> 
> 
> thanks a lot!!!
> 
> robert.
> 
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

This sounds like it could be dangerous, but if you're sure it's 
necessary and you know what you are doing, you could investigate whether 
the "pairlist" internal structure might enable you to do this (AFAIK,
a
"pairlist" is a traditional linked-list data structure).  In general, 
pairlists seem to be used very little, and I've never seen their use 
encouraged, so I would be very cautious.  You can read more about 
"pairlists" under ?pairlist, in the R-internals manual, and in the 
source starting at src/include/Rinternals.h (look for "LISTSXP").

-- Tony Plate

Robert Castelo wrote:
> dear people,
>
> i need to code a function in C working in R and receives two R SEXP
> objects as parameters, where one is a list and another is a vector of
> integers:
>
> void f(SEXP list, SEXP vector) {
>
>   ...
>
>   return list;
> }
>
>
> and it should return the given list with the integer vector concatenated
> at the end (as the last element of the list). the list can be really big
> so i would not like to create a new list from scratch and copy all the
> elements, including the additional one. i'm also interested in knowing
> how should i properly handle protections of the SEXP objects when doing
> this.
>
> i've been looking at the R source code for everything that has to do
> with CAR, CDR, CONS, and even found functions with promising names like
> listAppend or GrowList but i have not been able to figure this out nor i
> haven't been able to find any reference on how to do this by googling
> all around, so any help will be very much appreciated.
>
>
> thanks a lot!!!
>
> robert.
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
>

On Thu, 18 Oct 2007, Robert Castelo wrote:
> dear people,
>
> i need to code a function in C working in R and receives two R SEXP
> objects as parameters, where one is a list and another is a vector of
> integers:
>
> void f(SEXP list, SEXP vector) {
>
>  ...
>
>  return list;
> }
You are returning an result in a function that returns void: the compiler 
will complain at you.
> and it should return the given list with the integer vector concatenated
> at the end (as the last element of the list). the list can be really big
> so i would not like to create a new list from scratch and copy all the
> elements, including the additional one. i'm also interested in knowing
> how should i properly handle protections of the SEXP objects when doing
> this.
If you study the R Internals manual you will see that there is no space on 
the original VECSXP for another element, so you do *need* to create a new 
VECSXP.  Note that creating a new list does not mean necessarily copying 
the elements, but you do need to think about the NAMED bits.

If you are doing this repeatedly you could think about exploiting the 
TRUELENGTH field to create a list with spare space that you could exploit 
in future calls.

It is often possible to avoid copying, but considerable care is needed to 
ensure that you do not end up with an object that does not effectively 
share elements with another user-visible one.
> i've been looking at the R source code for everything that has to do
> with CAR, CDR, CONS, and even found functions with promising names like
> listAppend or GrowList but i have not been able to figure this out nor i
> haven't been able to find any reference on how to do this by googling
> all around, so any help will be very much appreciated.
Those are for pairlists, not (vector) lists.


-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595