R devel - Sep 2005 - as.data.frame segfaults on large lists (PR#8141)

Full_Name: Ulrich Poetter
Version: 2.1.1
OS: i686-pc-linux-gnu FC2
Submission from: (NULL) (134.147.95.187)


as.data.frame() segfaults on lists with very many elements:
> dfn <- rep(list(rep(0,2)),198000)
> test <- as.data.frame.list(dfn)
Process R segmentation fault at Sun Sep 18 17:06:02 2005

ulrich.poetter at ruhr-uni-bochum.de writes:
> Full_Name: Ulrich Poetter
> Version: 2.1.1
> OS: i686-pc-linux-gnu FC2
> Submission from: (NULL) (134.147.95.187)
> 
> 
> as.data.frame() segfaults on lists with very many elements:
> 
> > dfn <- rep(list(rep(0,2)),198000)
> > test <- as.data.frame.list(dfn)
> 
> Process R segmentation fault at Sun Sep 18 17:06:02 2005
Not for me on FC4. The process size grows to about 180M and the system
thrashes badly, but the calculation runs to completion.

It's not unlikely that we are ignoring a failed allocation somewhere,
but there's not much hope of finding it from the available
information. You could try running under gdb and see where things go
wrong for you.

-- 
   O__  ---- Peter Dalgaard             ?ster Farimagsgade 5, Entr.B
  c/ /'_ --- Dept. of Biostatistics     PO Box 2099, 1014 Cph. K
 (*) \(*) -- University of Copenhagen   Denmark          Ph:  (+45) 35327918
~~~~~~~~~~ - (p.dalgaard at biostat.ku.dk)                  FAX: (+45) 35327907

This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

---1903393524-540054921-1127222869=:28721
Content-Type: TEXT/PLAIN; charset=iso-8859-1; format=flowed
Content-Transfer-Encoding: QUOTED-PRINTABLE


Under Valgrind on x86_64 I get
=3D=3D27405=3D=3D  Access not within mapped region at address 0x33FFEFD8
=3D=3D27405=3D=3D    at 0x447045: Rf_substituteList (coerce.c:2003)
=3D=3D27405=3D=3D Stack overflow in thread 1: can't grow stack to 0x33FFEF98


 =09-thomas

On Sun, 18 Sep 2005, Peter Dalgaard wrote:
> ulrich.poetter at ruhr-uni-bochum.de writes:
>
>> Full_Name: Ulrich Poetter
>> Version: 2.1.1
>> OS: i686-pc-linux-gnu FC2
>> Submission from: (NULL) (134.147.95.187)
>>
>>
>> as.data.frame() segfaults on lists with very many elements:
>>
>>> dfn <- rep(list(rep(0,2)),198000)
>>> test <- as.data.frame.list(dfn)
>>
>> Process R segmentation fault at Sun Sep 18 17:06:02 2005
>
> Not for me on FC4. The process size grows to about 180M and the system
> thrashes badly, but the calculation runs to completion.
>
> It's not unlikely that we are ignoring a failed allocation somewhere,
> but there's not much hope of finding it from the available
> information. You could try running under gdb and see where things go
> wrong for you.
>
> --
>   O__  ---- Peter Dalgaard             =D8ster Farimagsgade 5, Entr.B
>  c/ /'_ --- Dept. of Biostatistics     PO Box 2099, 1014 Cph. K
> (*) \(*) -- University of Copenhagen   Denmark          Ph:  (+45) 35327918
> ~~~~~~~~~~ - (p.dalgaard at biostat.ku.dk)                  FAX: (+45)
35327907
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
Thomas Lumley=09=09=09Assoc. Professor, Biostatistics
tlumley at u.washington.edu=09University of Washington, Seattle
---1903393524-540054921-1127222869=:28721--

This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

---1903393524-2062424977-1127399140=:32170
Content-Type: TEXT/PLAIN; charset=X-UNKNOWN; format=flowed
Content-Transfer-Encoding: QUOTED-PRINTABLE


The problem is a deep recursion in substituteList, presumably from=20
substitute(list(...)) in data.frame().  This overflows the stack for large=20
enough lists.  Presumably the only solution is to rewrite substituteList=20
iteratively.

 =09-thomas

On Tue, 20 Sep 2005, Thomas Lumley wrote:
>
> Under Valgrind on x86_64 I get
> =3D=3D27405=3D=3D  Access not within mapped region at address 0x33FFEFD8
> =3D=3D27405=3D=3D    at 0x447045: Rf_substituteList (coerce.c:2003)
> =3D=3D27405=3D=3D Stack overflow in thread 1: can't grow stack to
0x33FFEF98
>
>
> =09-thomas
>
> On Sun, 18 Sep 2005, Peter Dalgaard wrote:
>
>> ulrich.poetter at ruhr-uni-bochum.de writes:
>>=20
>>> Full_Name: Ulrich Poetter
>>> Version: 2.1.1
>>> OS: i686-pc-linux-gnu FC2
>>> Submission from: (NULL) (134.147.95.187)
>>>=20
>>>=20
>>> as.data.frame() segfaults on lists with very many elements:
>>>=20
>>>> dfn <- rep(list(rep(0,2)),198000)
>>>> test <- as.data.frame.list(dfn)
>>>=20
>>> Process R segmentation fault at Sun Sep 18 17:06:02 2005
>>=20
>> Not for me on FC4. The process size grows to about 180M and the system
>> thrashes badly, but the calculation runs to completion.
>>=20
>> It's not unlikely that we are ignoring a failed allocation
somewhere,
>> but there's not much hope of finding it from the available
>> information. You could try running under gdb and see where things go
>> wrong for you.
>>=20
>> --
>>   O__  ---- Peter Dalgaard             =D8ster Farimagsgade 5, Entr.B
>>  c/ /'_ --- Dept. of Biostatistics     PO Box 2099, 1014 Cph. K
>> (*) \(*) -- University of Copenhagen   Denmark          Ph:  (+45)
35327918
>> ~~~~~~~~~~ - (p.dalgaard at biostat.ku.dk)                  FAX:
(+45)=20
>> 35327907
>>=20
>> ______________________________________________
>> R-devel at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>=20
>
> Thomas Lumley=09=09=09Assoc. Professor, Biostatistics
> tlumley at u.washington.edu=09University of Washington, Seattle
Thomas Lumley=09=09=09Assoc. Professor, Biostatistics
tlumley at u.washington.edu=09University of Washington, Seattle
---1903393524-2062424977-1127399140=:32170--