R devel - Aug 2013 - model.frame(), model.matrix(), and derived predictor variables

Dear r-developers:

  I am struggling with some fundamental aspects of model.frame().

  Conceptually, I think of a flow from data -> model.frame() ->
model.matrix; the data contain _input variables_, while model.matrix
contains _predictor variables_: data have been transformed, splines and
polynomials have been expanded into their corresponding
multi-dimensional bases, and factors have been expanded into appropriate
sets of dummy variables depending on their contrasts.
  I originally thought of model.frame() as containing input variables as
well (but with only the variables needed by the model, and with cases
containing NAs handled according to the relevant na.action setting), but
that's not quite true.  While factors are retained as-is, splines and
polynomials and parameter transformations are evaluated. For example

d <- data.frame(x=1:10,y=1:10)
model.frame(y~log(x),d)

produces a model frame with columns 'y', 'log(x)' (not
'y', 'x').

This makes it hard (impossible?) to use the model frame to re-evaluate
the existing formula in a model, e.g.

m <- lm(y~log(x),d)
update(m,data=model.frame(m))
## Error in eval(expr, envir, enclos) : object 'x' not found

It seems to me that this is a reasonable thing to want to do
(i.e. use the model frame as a stored copy of the data that
 can be used for additional model operations); otherwise, I
either need to carry along an additional copy of the data in
a slot, or hope that the model is still living in an environment
where it can find a copy of the original data.

Does anyone have any insights into the original design choices,
or suggestions about how they have handled this within their own
code? Do you just add an additional data slot to the model?  I've
considered trying to write some kind of 'augmented' model frame, that
would contain the equivalent of
setdiff(all.vars(formula),model.frame(m)) [i.e.  all input variables
that appeared in the formula but not in the model frame ...].

  thanks
   Ben Bolker

Bump: just trying one more time to see if anyone had thoughts on this
(so far it's just <crickets> ...)


-------- Original Message --------
Subject: model.frame(), model.matrix(), and derived predictor variables
Date: Sat, 17 Aug 2013 12:19:58 -0400
From: Ben Bolker <bbolker at gmail.com>
To: R-devel at stat.math.ethz.ch <R-devel at stat.math.ethz.ch>


  Dear r-developers:

  I am struggling with some fundamental aspects of model.frame().

  Conceptually, I think of a flow from data -> model.frame() ->
model.matrix; the data contain _input variables_, while model.matrix
contains _predictor variables_: data have been transformed, splines and
polynomials have been expanded into their corresponding
multi-dimensional bases, and factors have been expanded into appropriate
sets of dummy variables depending on their contrasts.
  I originally thought of model.frame() as containing input variables as
well (but with only the variables needed by the model, and with cases
containing NAs handled according to the relevant na.action setting), but
that's not quite true.  While factors are retained as-is, splines and
polynomials and parameter transformations are evaluated. For example

d <- data.frame(x=1:10,y=1:10)
model.frame(y~log(x),d)

produces a model frame with columns 'y', 'log(x)' (not
'y', 'x').

This makes it hard (impossible?) to use the model frame to re-evaluate
the existing formula in a model, e.g.

m <- lm(y~log(x),d)
update(m,data=model.frame(m))
## Error in eval(expr, envir, enclos) : object 'x' not found

It seems to me that this is a reasonable thing to want to do
(i.e. use the model frame as a stored copy of the data that
 can be used for additional model operations); otherwise, I
either need to carry along an additional copy of the data in
a slot, or hope that the model is still living in an environment
where it can find a copy of the original data.

Does anyone have any insights into the original design choices,
or suggestions about how they have handled this within their own
code? Do you just add an additional data slot to the model?  I've
considered trying to write some kind of 'augmented' model frame, that
would contain the equivalent of
setdiff(all.vars(formula),model.frame(m)) [i.e.  all input variables
that appeared in the formula but not in the model frame ...].

  thanks
   Ben Bolker