R help - Oct 2003 - problem using do.call and substitute for predict.glm using poly()

Hi

I am having a particular problem with some glm models I am running. I 
have been adapting code from Bill Venables 'Programmers niche' in RNews 
Vol 2/2 to fit ca. 1000 glm models to a combination of species 0/1 data 
(as Y) and related physicochemical data (X), to automate the process of 
fitting this many models. I have successfully managed to fit all the 
models and have stored the results in a list, each list has 47 main 
'branches' (one for each species) and each branch has 23
'leaves' that
each contain a glm object

But R throws up the following error:

Error in poly(Alk1, degree = 2, coefs = structure(list(alpha = 
c(37.7515662650602,  :
         Object "Alk1" not found

When trying to evaluate the following code:

pAsgn <- paste("predList[[i]][[n]] <- try(predict(resList$Y$X,
newdata
= 	data.frame(X = predData$X), type = 'response', se = TRUE))")
pAsgn <- parse(text = pAsgn)[[1]]
for (i in namY) {
     for (n in namX) {
         TAsgn <- do.call("substitute", list(pAsgn, list(n = n, i 
= i, X 			= as.name(n), Y = as.name(i))))
         eval(TAsgn)
     }
}

Alk1 is used above as an example, all 23 predictors are 'not found' 
depending on which part of the loop I'm in. Investigation of the 
predList object after this has run shows for example:

$Unk.nown$NCR
[1] "Error in poly(NCR, degree = 2, coefs = structure(list(alpha = 
c(218.156626506024,  : \n    Object \"NCR\" not found\n"
attr(,"class")
[1] "try-error"

pAsgn contains a parsed R expression:

predList[[i]][[n]] <- try(predict(resList$Y$X, newdata = 
data.frame(X = 		predData$X), type = "response", se = TRUE))

I think I have narrowed the problem down to the fact that the first X in 
newdata = data.frame(X = predData$X)... is not being substitute with the 
variable in question, where as all the other X and Y's are being 
substituted:
(n and i would be supplied by for loops (see above) so I have 
substituted  two values below as if they had been in the loop)

 > do.call("substitute", list(pAsgn, list(n = namX[1], i = 
namY[1], X = 			as.name(n), Y = as.name(i))))
predList[["Acr.harp"]][["Alk1"]] <-
try(predict(resList$Unk.nown$NCR,
     newdata = data.frame(X = predData$NCR), type = "response",
     se = TRUE))        ^^^^^^ problem here

If i supply the values I want for one of the runs, such as:

 > predList[[1]][[1]] <- try(predict(resList$Acr.harp$Alk1, newdata = 
data.frame(Alk1 = predData$Alk1), type = "response", se = TRUE))

Then this works, so the question is, how to I get X to be substituted in 
the above call? Perhaps this is not the cause of the error, so if anyone 
else has other suggestions.

Thank you for help.

Gav

ps: version
platform i386-pc-mingw32
arch     i386
os       mingw32
system   i386, mingw32
status
major    1
minor    8.0
year     2003
month    10
day      08
language R
-- 
%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%
Gavin Simpson                     [T] +44 (0)20 7679 5522
ENSIS Research Fellow             [F] +44 (0)20 7679 7565
ENSIS Ltd. & ECRC                 [E] gavin.simpson at ucl.ac.uk
UCL Department of Geography       [W] http://www.ucl.ac.uk/~ucfagls/cv/
26 Bedford Way                    [W] http://www.ucl.ac.uk/~ucfagls/
London.  WC1H 0AP.
%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%

On Mon, 27 Oct 2003, Gavin Simpson wrote:
>
> But R throws up the following error:
>
> Error in poly(Alk1, degree = 2, coefs = structure(list(alpha >
c(37.7515662650602,  :
>          Object "Alk1" not found
>
> When trying to evaluate the following code:
>
> pAsgn <- paste("predList[[i]][[n]] <- try(predict(resList$Y$X,
newdata
> = 	data.frame(X = predData$X), type = 'response', se =
TRUE))")
> pAsgn <- parse(text = pAsgn)[[1]]
> for (i in namY) {
>      for (n in namX) {
>          TAsgn <- do.call("substitute", list(pAsgn, list(n =
n, i
> = i, X 			= as.name(n), Y = as.name(i))))
>          eval(TAsgn)
>      }
> }
>
> Alk1 is used above as an example, all 23 predictors are 'not found'
> depending on which part of the loop I'm in. Investigation of the
> predList object after this has run shows for example:
>
> $Unk.nown$NCR
> [1] "Error in poly(NCR, degree = 2, coefs = structure(list(alpha >
c(218.156626506024,  : \n    Object \"NCR\" not found\n"
> attr(,"class")
> [1] "try-error"
>
> pAsgn contains a parsed R expression:
>
> predList[[i]][[n]] <- try(predict(resList$Y$X, newdata > data.frame(X
= 		predData$X), type = "response", se = TRUE))
>
> I think I have narrowed the problem down to the fact that the first X in
> newdata = data.frame(X = predData$X)... is not being substitute with the
> variable in question, where as all the other X and Y's are being
> substituted:
Yes, that's right.
> (n and i would be supplied by for loops (see above) so I have
> substituted  two values below as if they had been in the loop)
>
>  > do.call("substitute", list(pAsgn, list(n = namX[1], i >
namY[1], X = 			as.name(n), Y = as.name(i))))
> predList[["Acr.harp"]][["Alk1"]] <-
try(predict(resList$Unk.nown$NCR,
>      newdata = data.frame(X = predData$NCR), type = "response",
>      se = TRUE))        ^^^^^^ problem here
>
> If i supply the values I want for one of the runs, such as:
>
>  > predList[[1]][[1]] <- try(predict(resList$Acr.harp$Alk1, newdata
> data.frame(Alk1 = predData$Alk1), type = "response", se = TRUE))
>
> Then this works, so the question is, how to I get X to be substituted in
> the above call? Perhaps this is not the cause of the error, so if anyone
> else has other suggestions.

You should be able to use
   newdata=predData[n]
which would substitute to
   newdata=predData["Alk1"]
or even just
   newdata=predData

If you actually needed to substitute the tags you would (I think) need to
work with the parsed expression directly, which is possible but icky:
> names(pAsgn[[3]][[2]][[3]])[2]<-"Alk1"
> pAsgn
predList[[i]][[n]] <- try(predict(resList$Y$X, newdata = data.frame(Alk1 =
predData$X),
    type = "response", se = TRUE))


	-thomas

Dear List,

I think I have found the source of my problem in a reply from Thomas 
Lumley to a previous question on R-Help: 
http://www.r-project.org/nocvs/mail/r-help/2002/0586.html

My code is not working because substitute() does not substitute formal 
arguments to functions, and I guess the first X in data.frame(X = 
predData$X) is a formal argument to data.frame().

The recommended options are to use formals() or parse(deparse()) to 
achieve the required effect, but here I get a somewhat stuck.

I was wondering if anyone on the list could show my how to modify my 
existing code:

pAsgn <- paste("predList[[i]][[n]] <- try(predict(resList$Y$X,
newdata
= 	data.frame(X = predData$X), type = 'response', se = TRUE))")
pAsgn <- parse(text = pAsgn)[[1]]
for (i in namY) {
     for (n in namX) {
         TAsgn <- do.call("substitute", list(pAsgn, list(n = n, i =
i, X
		= as.name(n), Y = as.name(i))))
         eval(TAsgn)
     }
}

so that data.frame(X = predData$X) is replaced with the value of X such 
that the output from do.call in the loop is something like:

predList[["Acr.harp"]][["Alk1"]] <-
try(predict(resList$Acr.harp$Alk1,
     newdata = data.frame(Alk1 = predData$Alk1), type = "response",
     se = TRUE))

If anyone could point me in the right direction it would be most 
appreciated.

All the best

Gav

Gavin Simpson wrote:
> Hi
> 
> I am having a particular problem with some glm models I am running. I 
> have been adapting code from Bill Venables 'Programmers niche' in
RNews
> Vol 2/2 to fit ca. 1000 glm models to a combination of species 0/1 data 
> (as Y) and related physicochemical data (X), to automate the process of 
> fitting this many models. I have successfully managed to fit all the 
> models and have stored the results in a list, each list has 47 main 
> 'branches' (one for each species) and each branch has 23
'leaves' that
> each contain a glm object
> 
> But R throws up the following error:
> 
> Error in poly(Alk1, degree = 2, coefs = structure(list(alpha = 
> c(37.7515662650602,  :
>         Object "Alk1" not found
> 
> When trying to evaluate the following code:
> 
> pAsgn <- paste("predList[[i]][[n]] <- try(predict(resList$Y$X,
newdata
> =     data.frame(X = predData$X), type = 'response', se =
TRUE))")
> pAsgn <- parse(text = pAsgn)[[1]]
> for (i in namY) {
>     for (n in namX) {
>         TAsgn <- do.call("substitute", list(pAsgn, list(n = n,
i = i,
> X             = as.name(n), Y = as.name(i))))
>         eval(TAsgn)
>     }
> }
> 
> Alk1 is used above as an example, all 23 predictors are 'not found'
> depending on which part of the loop I'm in. Investigation of the 
> predList object after this has run shows for example:
> 
> $Unk.nown$NCR
> [1] "Error in poly(NCR, degree = 2, coefs = structure(list(alpha = 
> c(218.156626506024,  : \n    Object \"NCR\" not found\n"
> attr(,"class")
> [1] "try-error"
> 
> pAsgn contains a parsed R expression:
> 
> predList[[i]][[n]] <- try(predict(resList$Y$X, newdata = data.frame(X 
> =         predData$X), type = "response", se = TRUE))
> 
> I think I have narrowed the problem down to the fact that the first X in 
> newdata = data.frame(X = predData$X)... is not being substitute with the 
> variable in question, where as all the other X and Y's are being 
> substituted:
> (n and i would be supplied by for loops (see above) so I have 
> substituted  two values below as if they had been in the loop)
> 
>  > do.call("substitute", list(pAsgn, list(n = namX[1], i =
namY[1], X
> =             as.name(n), Y = as.name(i))))
> predList[["Acr.harp"]][["Alk1"]] <-
try(predict(resList$Unk.nown$NCR,
>     newdata = data.frame(X = predData$NCR), type = "response",
>     se = TRUE))        ^^^^^^ problem here
> 
> If i supply the values I want for one of the runs, such as:
> 
>  > predList[[1]][[1]] <- try(predict(resList$Acr.harp$Alk1, newdata =
> data.frame(Alk1 = predData$Alk1), type = "response", se = TRUE))
> 
> Then this works, so the question is, how to I get X to be substituted in 
> the above call? Perhaps this is not the cause of the error, so if anyone 
> else has other suggestions.
> 
> Thank you for help.
> 
> Gav
> 
> ps: version
> platform i386-pc-mingw32
> arch     i386
> os       mingw32
> system   i386, mingw32
> status
> major    1
> minor    8.0
> year     2003
> month    10
> day      08
> language R
-- 
%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%
Gavin Simpson                     [T] +44 (0)20 7679 5522
ENSIS Research Fellow             [F] +44 (0)20 7679 7565
ENSIS Ltd. & ECRC                 [E] gavin.simpson at ucl.ac.uk
UCL Department of Geography       [W] http://www.ucl.ac.uk/~ucfagls/cv/
26 Bedford Way                    [W] http://www.ucl.ac.uk/~ucfagls/
London.  WC1H 0AP.
%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%