R devel - Oct 2010 - model.frame deficiency

The model.frame function has trouble with a certain type of really long
formula.  Here is a test:

tname <- paste('var', 1:50, sep='')
tmat <- matrix(rnorm(500), ncol=50, dimnames=list(NULL, tname))
tdata <- data.frame(tmat)

temp1 <- paste( paste(tname, tname, sep='='), collapse=', ')
temp2 <- paste("~1 + cbind(", temp1, ")")
model.frame(as.formula(temp2), tdata)

 This came out of a user report to me.  Coxph with a very long formula
(which does work) followed by a call to survexp leads to a call like the
above but using my "ratetable" function instead of "cbind".

My machine's details:
R version 2.10.0 (2009-10-26) 
x86_64-unknown-linux-gnu 

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=C              
 [5] LC_MONETARY=C              LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] grDevices datasets  splines   graphics  utils     stats
methods  
[8] base     


Editorial comment: I said "deficiency" not "bug" above, as
I'm not so
sure how good a model with 75 variables might be in the first place.
(Though in this case the user is pretty savvy.)  

Terry Therneau

At least for me, the problem is not to do with the length of 
formula.  You have a term in the formula with a very long name, and 
there are several places that can go wrong.  For example, for the N=20 
version I get
> model.frame(as.formula(temp2), tdata) -> foo
> foo
Error in do.call("data.frame", rval) :
   variable names are limited to 256 bytes

For the N=50 case the sticking point is

     varnames <- sapply(vars, deparse, width.cutoff = 500)[-1L]

which could be rewritten as

     varnames <- sapply(vars, function(x) paste(deparse(x,width.cutoff=500),
collapse=' '))[-1L]

but you would still not be able to print the result.


On Thu, 7 Oct 2010, Terry Therneau wrote:
> The model.frame function has trouble with a certain type of really long
> formula.  Here is a test:
>
> tname <- paste('var', 1:50, sep='')
> tmat <- matrix(rnorm(500), ncol=50, dimnames=list(NULL, tname))
> tdata <- data.frame(tmat)
>
> temp1 <- paste( paste(tname, tname, sep='='), collapse=',
')
> temp2 <- paste("~1 + cbind(", temp1, ")")
> model.frame(as.formula(temp2), tdata)
>
> This came out of a user report to me.  Coxph with a very long formula
> (which does work) followed by a call to survexp leads to a call like the
> above but using my "ratetable" function instead of
"cbind".
>
> My machine's details:
> R version 2.10.0 (2009-10-26)
> x86_64-unknown-linux-gnu
>
> locale:
> [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
> [3] LC_TIME=en_US.UTF-8        LC_COLLATE=C
> [5] LC_MONETARY=C              LC_MESSAGES=en_US.UTF-8
> [7] LC_PAPER=en_US.UTF-8       LC_NAME=C
> [9] LC_ADDRESS=C               LC_TELEPHONE=C
> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
>
> attached base packages:
> [1] grDevices datasets  splines   graphics  utils     stats
> methods
> [8] base
>
>
> Editorial comment: I said "deficiency" not "bug" above,
as I'm not so
> sure how good a model with 75 variables might be in the first place.
> (Though in this case the user is pretty savvy.)
>
> Terry Therneau
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595