R help - Jan 2011 - Meaning of pterms in survreg object?

I am trying to model survival data with a Weibull distribution
using survreg. Units are clustered two apiece, sometimes receiving
the same treatment and sometimes opposing treatment.

Residual and predict methods are not carried out on the survreg
object, although the component linear.predictors exists for the
survreg object. Looking at the code I see that the residual method
refuses to run if any of the components of the pterms vector is
equal to 2.

When taking the logarithm of the data and trying a linear mixed
model, residuals and predicted values are produced just fine.

So what is pterms and why is it that, when any component of it
is 2, residuals are not allowed to be displayed?     
-- 
Johannes H?sing               There is something fascinating about science. 
                              One gets such wholesale returns of conjecture 
mailto:johannes at huesing.name  from such a trifling investment of fact.
http://derwisch.wikidot.com         (Mark Twain, "Life on the
Mississippi")

I'm having a hard time fathoming your question, since "pterms"
appears
nowhere in my source code to residuals.survreg.  Can we start at the
beginning, per the posting guide?

 1. What version of R are you running?
 2. What exactly are the statements you typed which give the message?

Note that residuals are not saved in a fit, but are computed later.
  fit <- survreg(Surv(time, status) ~ age + height, data=mydata)
  resid(fit, type='response')

Terry Therneau

--- begin included message --------
Residual and predict methods are not carried out on the
survreg                                                                        
object, although the component linear.predictors exists for
the                                                                        
survreg object. Looking at the code I see that the residual
method                                                                     
refuses to run if any of the components of the pterms vector
is                                                                        
equal to
2.
--- end inclusion ---

Is there a special mailing list for the survival package where my
question would have been more appropriate?

Johannes Huesing <johannes at huesing.name> [Mon, Jan 10, 2011 at
09:26:37PM CET]:
> I am trying to model survival data with a Weibull distribution
> using survreg. Units are clustered two apiece, sometimes receiving
> the same treatment and sometimes opposing treatment.
>
> Residual and predict methods are not carried out on the survreg
> object, although the component linear.predictors exists for the
> survreg object. Looking at the code I see that the residual method
> refuses to run if any of the components of the pterms vector is
> equal to 2.
>
> When taking the logarithm of the data and trying a linear mixed
> model, residuals and predicted values are produced just fine.
>
> So what is pterms and why is it that, when any component of it
> is 2, residuals are not allowed to be displayed?     
> -- 
> Johannes H?sing               There is something fascinating about science.
>                               One gets such wholesale returns of conjecture
> mailto:johannes at huesing.name  from such a trifling investment of fact.
> http://derwisch.wikidot.com         (Mark Twain, "Life on the
Mississippi")
> 
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
-- 
Johannes H?sing               There is something fascinating about science. 
                              One gets such wholesale returns of conjecture 
mailto:johannes at huesing.name  from such a trifling investment of fact.
http://derwisch.wikidot.com         (Mark Twain, "Life on the
Mississippi")

Johannes Huesing <johannes at huesing.name> [Mon, Jan 10, 2011 at
09:26:37PM CET]:
> I am trying to model survival data with a Weibull distribution
> using survreg. Units are clustered two apiece, sometimes receiving
> the same treatment and sometimes opposing treatment.
>
> Residual and predict methods are not carried out on the survreg
> object, although the component linear.predictors exists for the
> survreg object. Looking at the code I see that the residual method
> refuses to run if any of the components of the pterms vector is
> equal to 2.
>
> When taking the logarithm of the data and trying a linear mixed
> model, residuals and predicted values are produced just fine.
>
> So what is pterms and why is it that, when any component of it
> is 2, residuals are not allowed to be displayed?     
I am using R 2.10.1, and here is some example code:

anzpat <- 32
device <- rep(unlist(expand.grid(list(c("C", "Z"),
c("C", "Z")))),
              anzpat / 4)
bleedtimes <- exp(rnorm(anzpat * 2, 0, .3) + as.numeric(device) * log(3))
pat <- as.factor(rep(1:anzpat, each=2))

library(survival)

survmod <- survreg(Surv(sapply(bleedtimes, min, 10), bleedtimes < 10) ~
device + frailty.gaussian(pat))

residuals(survmod)

###  Fehler in residuals.survreg.penal(survmod) : 
###  Residualss not available for sparse models

survmod$pterms

###          (Intercept)                device frailty.gaussian(pat) 
###                    0                     0                     2 


-- 
Johannes H?sing               There is something fascinating about science. 
                              One gets such wholesale returns of conjecture 
mailto:johannes at huesing.name  from such a trifling investment of fact.
http://derwisch.wikidot.com         (Mark Twain, "Life on the
Mississippi")

> survmod <- survreg(Surv(sapply(bleedtimes, min, 10), bleedtimes < 10)
>       ~ device + frailty.gaussian(pat))
> residuals(survmod)
> Error in residuals.survreg.penal(survmod) : 
>   Residuals not available for sparse models
 Thanks for the reproducable error.  

 a. With respect to your earliest question, the "pterms" attributes in
a
survreg model are used to keep track of penalized terms.  Smoothing
splines and frailties are implimented in this way.

 b. A gaussian frailty with the default "reml" method does not work
correctly with survreg.  See the manual page help('frailty').  

 c. You could use frailty.gaussian(pat, sparse=1000, method='aic').  The
first argument avoids the use of a sparse approximation (your example
had 32 patients which is < 1000), the second uses AIC to choose the size
of the random effect.
   I had forgotton that extension of the residuals to the sparse case
was an addition that I never finished.  With 100 or fewer groups the
full computation is not usually a burden, the sparse matrix
approximations were mostly designed for genetics studies where I have
thousands of subjects.

Terry Therneau