R help - Nov 2012 - as.data.frame(do.call(rbind,lapply)) produces something weird

The following code:
--8<---------------cut
here---------------start------------->8---
> myfun <- function (x) list(x=x,y=x*x)
> z <- as.data.frame(do.call(rbind,lapply(1:3,function(x)
c(a=paste("a",x,sep=""),as.list(unlist(list(b=myfun(x),c=myfun(x*x*x))))))))
> z
   a b.x b.y c.x c.y
1 a1   1   1   1   1
2 a2   2   4   8  64
3 a3   3   9  27 729
--8<---------------cut here---------------end--------------->8---
the appearance of z is good, but str() and summary betray some weirdness:
--8<---------------cut
here---------------start------------->8---
> str(z)
'data.frame':	3 obs. of  5 variables:
 $ a  :List of 3
  ..$ : chr "a1"
  ..$ : chr "a2"
  ..$ : chr "a3"
 $ b.x:List of 3
  ..$ : int 1
  ..$ : int 2
  ..$ : int 3
 $ b.y:List of 3
  ..$ : int 1
  ..$ : int 4
  ..$ : int 9
 $ c.x:List of 3
  ..$ : int 1
  ..$ : int 8
  ..$ : int 27
 $ c.y:List of 3
  ..$ : int 1
  ..$ : int 64
  ..$ : int 729
--8<---------------cut here---------------end--------------->8---
how do I ensure that the columns of z are vectors, as in
--8<---------------cut
here---------------start------------->8---
> z <-
data.frame(a=c("a1","a2","a3"),b.x=c(1,2,3),b.y=c(1,4,9),c.x=c(1,8,27),c.y=c(1,64,729))
> z
   a b.x b.y c.x c.y
1 a1   1   1   1   1
2 a2   2   4   8  64
3 a3   3   9  27 729
> str(z)
'data.frame':	3 obs. of  5 variables:
 $ a  : Factor w/ 3 levels "a1","a2","a3": 1 2 3
 $ b.x: num  1 2 3
 $ b.y: num  1 4 9
 $ c.x: num  1 8 27
 $ c.y: num  1 64 729
--8<---------------cut here---------------end--------------->8---
thanks!

-- 
Sam Steingold (http://sds.podval.org/) on Ubuntu 12.04 (precise) X 11.0.11103000
http://www.childpsy.net/ http://jihadwatch.org http://think-israel.org
http://www.PetitionOnline.com/tap12009/ http://honestreporting.com
Programming is like sex: one mistake and you have to support it for a lifetime.

Hi,
May be this helps:
z1<-as.data.frame(do.call(rbind,lapply(1:3,function(x)
c(a=paste("a",x,sep=""),unlist(do.call(c,list(b=myfun(x),c=myfun(x*x*x))))))))
z2<-within(z1,{b.x<-as.numeric(as.character(b.x));b.y<-as.numeric(as.character(b.y));c.x<-as.numeric(as.character(c.x));c.y<-as.numeric(as.character(c.y))})
?str(z2)
#'data.frame':??? 3 obs. of? 5 variables:
# $ a? : Factor w/ 3 levels "a1","a2","a3": 1 2 3
# $ b.x: num? 1 2 3
# $ b.y: num? 1 4 9
# $ c.x: num? 1 8 27
# $ c.y: num? 1 64 729


?z2
#?? a b.x b.y c.x c.y
#1 a1?? 1?? 1?? 1?? 1
#2 a2?? 2?? 4?? 8? 64
#3 a3?? 3?? 9? 27 729
A.K.



----- Original Message -----
From: Sam Steingold <sds at gnu.org>
To: r-help at r-project.org
Cc: 
Sent: Friday, November 9, 2012 2:21 PM
Subject: [R] as.data.frame(do.call(rbind,lapply)) produces something weird

The following code:
--8<---------------cut
here---------------start------------->8---
> myfun <- function (x) list(x=x,y=x*x)
> z <- as.data.frame(do.call(rbind,lapply(1:3,function(x)
c(a=paste("a",x,sep=""),as.list(unlist(list(b=myfun(x),c=myfun(x*x*x))))))))
> z
?  a b.x b.y c.x c.y
1 a1?  1?  1?  1?  1
2 a2?  2?  4?  8? 64
3 a3?  3?  9? 27 729
--8<---------------cut here---------------end--------------->8---
the appearance of z is good, but str() and summary betray some weirdness:
--8<---------------cut
here---------------start------------->8---
> str(z)
'data.frame':??? 3 obs. of? 5 variables:
$ a? :List of 3
? ..$ : chr "a1"
? ..$ : chr "a2"
? ..$ : chr "a3"
$ b.x:List of 3
? ..$ : int 1
? ..$ : int 2
? ..$ : int 3
$ b.y:List of 3
? ..$ : int 1
? ..$ : int 4
? ..$ : int 9
$ c.x:List of 3
? ..$ : int 1
? ..$ : int 8
? ..$ : int 27
$ c.y:List of 3
? ..$ : int 1
? ..$ : int 64
? ..$ : int 729
--8<---------------cut here---------------end--------------->8---
how do I ensure that the columns of z are vectors, as in
--8<---------------cut
here---------------start------------->8---
> z <-
data.frame(a=c("a1","a2","a3"),b.x=c(1,2,3),b.y=c(1,4,9),c.x=c(1,8,27),c.y=c(1,64,729))
> z
?  a b.x b.y c.x c.y
1 a1?  1?  1?  1?  1
2 a2?  2?  4?  8? 64
3 a3?  3?  9? 27 729
> str(z)
'data.frame':??? 3 obs. of? 5 variables:
$ a? : Factor w/ 3 levels "a1","a2","a3": 1 2 3
$ b.x: num? 1 2 3
$ b.y: num? 1 4 9
$ c.x: num? 1 8 27
$ c.y: num? 1 64 729
--8<---------------cut here---------------end--------------->8---
thanks!

-- 
Sam Steingold (http://sds.podval.org/) on Ubuntu 12.04 (precise) X 11.0.11103000
http://www.childpsy.net/ http://jihadwatch.org http://think-israel.org
http://www.PetitionOnline.com/tap12009/ http://honestreporting.com
Programming is like sex: one mistake and you have to support it for a lifetime.

______________________________________________
R-help at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Your call to rbind() creates matrix of mode "list".  Thus every
element
can be of a different type, although you "know" that there is a
pattern
to the types.  E.g.,
  > R <- rbind(
          list(Letter="a", Integer=1L, Complex=1+1i),
          list(Letter="b", Integer=2L, Complex=2+2i))
  > str(R)
  List of 6
   $ : chr "a"
   $ : chr "b"
   $ : int 1
   $ : int 2
   $ : cplx 1+1i
   $ : cplx 2+2i
   - attr(*, "dim")= int [1:2] 2 3
   - attr(*, "dimnames")=List of 2
    ..$ : NULL
    ..$ : chr [1:3] "Letter" "Integer" "Complex"

data.frame(R), since R is a matrix, will make a data.frame containing
the columns of R.  It does not decide that since each column is a list
that it should what data.frame(list(...)) would do, it just sticks those
columns, as is, into the data.frame that it creates:

  > Rdf <- data.frame(R)
  > str(Rdf)
  'data.frame':   2 obs. of  3 variables:
   $ Letter :List of 2
    ..$ : chr "a"
    ..$ : chr "b"
   $ Integer:List of 2
    ..$ : int 1
    ..$ : int 2
   $ Complex:List of 2
    ..$ : cplx 1+1i
    ..$ : cplx 2+2i

You can convert those columns to their "natural" type, at least
the type of their first element, with

  > for(i in seq_along(Rdf)) Rdf[[i]] <- as(Rdf[[i]],
class(Rdf[[i]][[1]]))
  > str(Rdf)
  'data.frame':   2 obs. of  3 variables:
   $ Letter : chr  "a" "b"
   $ Integer: int  1 2
   $ Complex: cplx  1+1i 2+2i

Note the as(list(...), atomicType) does the conversion if every element
of list(...) has length 1 and throws an error otherwise.  That is probably
a good check in this case.  unlist() would give the same result, perhaps
more quickly, if the list has the structure you expect but would silently
give bad results if some element of the list did not have length one.

Is that what you are looking for?

Note that storing things in a list takes a lot more memory  than storing
them as atomic vectors so your technique may not scale up very well.
  > object.size(as.list(1:1e6)) / object.size(1:1e6)
  13.9998700013 bytes

Bill Dunlap
Spotfire, TIBCO Software
wdunlap tibco.com

> -----Original Message-----
> From: r-help-bounces at r-project.org [mailto:r-help-bounces at
r-project.org] On Behalf
> Of Sam Steingold
> Sent: Friday, November 09, 2012 11:22 AM
> To: r-help at r-project.org
> Subject: [R] as.data.frame(do.call(rbind,lapply)) produces something weird
> 
> The following code:
> --8<---------------cut here---------------start------------->8---
> > myfun <- function (x) list(x=x,y=x*x)
> > z <- as.data.frame(do.call(rbind,lapply(1:3,function(x)
>
c(a=paste("a",x,sep=""),as.list(unlist(list(b=myfun(x),c=myfun(x*x*x))))))))
> > z
>    a b.x b.y c.x c.y
> 1 a1   1   1   1   1
> 2 a2   2   4   8  64
> 3 a3   3   9  27 729
> --8<---------------cut here---------------end--------------->8---
> the appearance of z is good, but str() and summary betray some weirdness:
> --8<---------------cut here---------------start------------->8---
> > str(z)
> 'data.frame':	3 obs. of  5 variables:
>  $ a  :List of 3
>   ..$ : chr "a1"
>   ..$ : chr "a2"
>   ..$ : chr "a3"
>  $ b.x:List of 3
>   ..$ : int 1
>   ..$ : int 2
>   ..$ : int 3
>  $ b.y:List of 3
>   ..$ : int 1
>   ..$ : int 4
>   ..$ : int 9
>  $ c.x:List of 3
>   ..$ : int 1
>   ..$ : int 8
>   ..$ : int 27
>  $ c.y:List of 3
>   ..$ : int 1
>   ..$ : int 64
>   ..$ : int 729
> --8<---------------cut here---------------end--------------->8---
> how do I ensure that the columns of z are vectors, as in
> --8<---------------cut here---------------start------------->8---
> > z <-
>
data.frame(a=c("a1","a2","a3"),b.x=c(1,2,3),b.y=c(1,4,9),c.x=c(1,8,27),c.y=c(1,64,729))
> > z
>    a b.x b.y c.x c.y
> 1 a1   1   1   1   1
> 2 a2   2   4   8  64
> 3 a3   3   9  27 729
> > str(z)
> 'data.frame':	3 obs. of  5 variables:
>  $ a  : Factor w/ 3 levels "a1","a2","a3": 1
2 3
>  $ b.x: num  1 2 3
>  $ b.y: num  1 4 9
>  $ c.x: num  1 8 27
>  $ c.y: num  1 64 729
> --8<---------------cut here---------------end--------------->8---
> thanks!
> 
> --
> Sam Steingold (http://sds.podval.org/) on Ubuntu 12.04 (precise) X
11.0.11103000
> http://www.childpsy.net/ http://jihadwatch.org http://think-israel.org
> http://www.PetitionOnline.com/tap12009/ http://honestreporting.com
> Programming is like sex: one mistake and you have to support it for a
lifetime.
> 
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

This is a case where you start by breaking down you complex one-liner
into separate statements and examine what the results are at each
point.  This is what I would have to do with the script you posted.

I think as Bill pointed out, one of your function calls is probably
creating a result that you were not expecting.  Standard (good)
programming practice would have you creating a number of simple
statements; this allows ease of debugging.

On Fri, Nov 9, 2012 at 2:21 PM, Sam Steingold <sds at gnu.org>
wrote:
> The following code:
> --8<---------------cut here---------------start------------->8---
>> myfun <- function (x) list(x=x,y=x*x)
>> z <- as.data.frame(do.call(rbind,lapply(1:3,function(x)
c(a=paste("a",x,sep=""),as.list(unlist(list(b=myfun(x),c=myfun(x*x*x))))))))
>> z
>    a b.x b.y c.x c.y
> 1 a1   1   1   1   1
> 2 a2   2   4   8  64
> 3 a3   3   9  27 729
> --8<---------------cut here---------------end--------------->8---
> the appearance of z is good, but str() and summary betray some weirdness:
> --8<---------------cut here---------------start------------->8---
>> str(z)
> 'data.frame':   3 obs. of  5 variables:
>  $ a  :List of 3
>   ..$ : chr "a1"
>   ..$ : chr "a2"
>   ..$ : chr "a3"
>  $ b.x:List of 3
>   ..$ : int 1
>   ..$ : int 2
>   ..$ : int 3
>  $ b.y:List of 3
>   ..$ : int 1
>   ..$ : int 4
>   ..$ : int 9
>  $ c.x:List of 3
>   ..$ : int 1
>   ..$ : int 8
>   ..$ : int 27
>  $ c.y:List of 3
>   ..$ : int 1
>   ..$ : int 64
>   ..$ : int 729
> --8<---------------cut here---------------end--------------->8---
> how do I ensure that the columns of z are vectors, as in
> --8<---------------cut here---------------start------------->8---
>> z <-
data.frame(a=c("a1","a2","a3"),b.x=c(1,2,3),b.y=c(1,4,9),c.x=c(1,8,27),c.y=c(1,64,729))
>> z
>    a b.x b.y c.x c.y
> 1 a1   1   1   1   1
> 2 a2   2   4   8  64
> 3 a3   3   9  27 729
>> str(z)
> 'data.frame':   3 obs. of  5 variables:
>  $ a  : Factor w/ 3 levels "a1","a2","a3": 1
2 3
>  $ b.x: num  1 2 3
>  $ b.y: num  1 4 9
>  $ c.x: num  1 8 27
>  $ c.y: num  1 64 729
> --8<---------------cut here---------------end--------------->8---
> thanks!
>
> --
> Sam Steingold (http://sds.podval.org/) on Ubuntu 12.04 (precise) X
11.0.11103000
> http://www.childpsy.net/ http://jihadwatch.org http://think-israel.org
> http://www.PetitionOnline.com/tap12009/ http://honestreporting.com
> Programming is like sex: one mistake and you have to support it for a
lifetime.
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.


-- 
Jim Holtman
Data Munger Guru

What is the problem that you are trying to solve?
Tell me what you want to do, not how you want to do it.

On Nov 9, 2012, at 11:21 AM, Sam Steingold wrote:
> The following code:
> --8<---------------cut here---------------start------------->8---
>> myfun <- function (x) list(x=x,y=x*x)
>> z <- as.data.frame(do.call(rbind,lapply(1:3,function(x) 
Winsemius' Corollary:

Kurt G?del's Incomprehension Theorem (one of his less well-known results) 
has as one of its corollaries that writing `rbind` or `cbind` inside
`data.frame` or `as.data.frame`  has probability of measure 1 of producing a
result that will dissatisfy its author.

-- 
David.

>>
c(a=paste("a",x,sep=""),as.list(unlist(list(b=myfun(x),c=myfun(x*x*x))))))))
>> z
>   a b.x b.y c.x c.y
> 1 a1   1   1   1   1
> 2 a2   2   4   8  64
> 3 a3   3   9  27 729
> --8<---------------cut here---------------end--------------->8---
> the appearance of z is good, but str() and summary betray some weirdness:
> --8<---------------cut here---------------start------------->8---
>> str(z)
> 'data.frame':	3 obs. of  5 variables:
> $ a  :List of 3
>  ..$ : chr "a1"
>  ..$ : chr "a2"
>  ..$ : chr "a3"
> $ b.x:List of 3
>  ..$ : int 1
>  ..$ : int 2
>  ..$ : int 3
> $ b.y:List of 3
>  ..$ : int 1
>  ..$ : int 4
>  ..$ : int 9
> $ c.x:List of 3
>  ..$ : int 1
>  ..$ : int 8
>  ..$ : int 27
> $ c.y:List of 3
>  ..$ : int 1
>  ..$ : int 64
>  ..$ : int 729
> --8<---------------cut here---------------end--------------->8---
> how do I ensure that the columns of z are vectors, as in
> --8<---------------cut here---------------start------------->8---
>> z <-
data.frame(a=c("a1","a2","a3"),b.x=c(1,2,3),b.y=c(1,4,9),c.x=c(1,8,27),c.y=c(1,64,729))
>> z
>   a b.x b.y c.x c.y
> 1 a1   1   1   1   1
> 2 a2   2   4   8  64
> 3 a3   3   9  27 729
>> str(z)
> 'data.frame':	3 obs. of  5 variables:
> $ a  : Factor w/ 3 levels "a1","a2","a3": 1 2
3
> $ b.x: num  1 2 3
> $ b.y: num  1 4 9
> $ c.x: num  1 8 27
> $ c.y: num  1 64 729
> --8<---------------cut here---------------end--------------->8---
> thanks!
> 
> -- 
> Sam Steingold (http://sds.podval.org/) on Ubuntu 12.04 (precise) X
11.0.11103000
> http://www.childpsy.net/ http://jihadwatch.org http://think-israel.org
> http://www.PetitionOnline.com/tap12009/ http://honestreporting.com
> Programming is like sex: one mistake and you have to support it for a
lifetime.
> 
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
David Winsemius, MD
Alameda, CA, USA