Hello all, I am trying to understand how to specify nested factors when using coxph(), and if it is appropriate to nest these factors in my situation. In the simplest form, I am testing two different temperatures, with each temperature being performed twice in different experimental periods (e.g. Temp5 performed in Period A and C, Temp4 performed in Period B and D) I am trying to see if survival time is affected by the treatment temperature. To do this I am using temperature and experimental period nested within temperature as factors.> LOEtempmod.5days=coxph(LOE.stable.5days~Temp+Temp/Period,data=goodstable)Warning message: In coxph(LOE.stable.5days ~ Temp + Temp/Period, : X matrix deemed to be singular; variable 2 5 6 7 1. Is this an appropriate way of nesting? 2. Can this error message be ignored? Any help would be much appreciated, thank you in advance. Katie
On Jan 11, 2013, at 5:35 PM, Katie Anweiler wrote:> Hello all, > I am trying to understand how to specify nested factors when using > coxph(), and if it is appropriate to nest these factors in my > situation. > In the simplest form, I am testing two different temperatures, with > each temperature being performed twice in different experimental > periods (e.g. Temp5 performed in Period A and C, Temp4 performed in > Period B and D) > I am trying to see if survival time is affected by the treatment > temperature. To do this I am using temperature and experimental > period nested within temperature as factors. > >> LOEtempmod.5days=coxph(LOE.stable.5days~Temp+Temp/ >> Period,data=goodstable) > Warning message: > In coxph(LOE.stable.5days ~ Temp + Temp/Period, : > X matrix deemed to be singular; variable 2 5 6 7 > > 1. Is this an appropriate way of nesting?Have you looked at the coxme package? http://cran.r-project.org/web/packages/coxme/index.html> 2. Can this error message be ignored?Sometimes R packages correctly drop variables that are exactly collinear: other times the correct solution is not clear. I would think the answer in this case would be "no", but do not have a lot to go on at this point. -- David Winsemius, MD Alameda, CA, USA
Katie: You need to get local statistical help. What you are doing makes no sense. See inline below. -- Bert On Sat, Jan 12, 2013 at 1:03 PM, David Winsemius <dwinsemius at comcast.net> wrote:> > On Jan 11, 2013, at 5:35 PM, Katie Anweiler wrote: > >> Hello all, >> I am trying to understand how to specify nested factors when using >> coxph(), and if it is appropriate to nest these factors in my >> situation. >> In the simplest form, I am testing two different temperatures, with >> each temperature being performed twice in different experimental >> periods (e.g. Temp5 performed in Period A and C, Temp4 performed in >> Period B and D)Period is confounded with temperature. That is the source of the singularity. in the message received below. You can estimate the C-A and the D-B differences. As I said, get statistical help. These are not R questions. -- Bert>> I am trying to see if survival time is affected by the treatment >> temperature. To do this I am using temperature and experimental >> period nested within temperature as factors. >> >>> LOEtempmod.5days=coxph(LOE.stable.5days~Temp+Temp/Period,data=goodstable) >> >> Warning message: >> In coxph(LOE.stable.5days ~ Temp + Temp/Period, : >> X matrix deemed to be singular; variable 2 5 6 7 >> >> 1. Is this an appropriate way of nesting? > > > Have you looked at the coxme package? > http://cran.r-project.org/web/packages/coxme/index.html > >> 2. Can this error message be ignored? > > > Sometimes R packages correctly drop variables that are exactly collinear: > other times the correct solution is not clear. I would think the answer in > this case would be "no", but do not have a lot to go on at this point. > > -- > > David Winsemius, MD > Alameda, CA, USA > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code.-- Bert Gunter Genentech Nonclinical Biostatistics Internal Contact Info: Phone: 467-7374 Website: http://pharmadevelopment.roche.com/index/pdb/pdb-functional-groups/pdb-biostatistics/pdb-ncb-home.htm
Thank you for the suggestions. Just to clarify, my first question was more on what actual coding I should be using to indicate a nested variable when using the coxph() function. I asked this after consulting several times with a local statistician, but unfortunately neither of us are very familiar with R. After further consultation, I have changed the design to a 2*2 design (2 levels of ExpTemp and Stability each) with blocking (Period). I am still getting the "x matrix deemed to be singular" error.> LOEmod3alt=coxph(LOE.fit~ExpTemp+Stability+Period,data=goodexp)Warning message: In coxph(LOE.fit ~ ExpTemp + Stability + Period, data = goodexp) : X matrix deemed to be singular; variable 5> summary(LOEmod3alt)Call: coxph(formula = LOE.fit ~ ExpTemp + Stability + Period, data = goodexp) n= 184, number of events= 105 coef exp(coef) se(coef) z Pr(>|z|) ExpTemp -3.17825 0.04166 0.53105 -5.985 2.17e-09 *** StabilityStatic -0.84129 0.43115 0.20470 -4.110 3.96e-05 *** PeriodB 1.06794 2.90937 0.22859 4.672 2.98e-06 *** PeriodC 1.23853 3.45054 0.58457 2.119 0.0341 * PeriodD NA NA 0.00000 NA NA --- Signif. codes: 0 ?***? 0.001 ?**? 0.01 ?*? 0.05 ?.? 0.1 ? ? 1 exp(coef) exp(-coef) lower .95 upper .95 ExpTemp 0.04166 24.0047 0.01471 0.118 StabilityStatic 0.43115 2.3194 0.28866 0.644 PeriodB 2.90937 0.3437 1.85877 4.554 PeriodC 3.45054 0.2898 1.09723 10.851 PeriodD NA NA NA NA Concordance= 0.833 (se = 0.03 ) Rsquare= 0.591 (max possible= 0.995 ) Likelihood ratio test= 164.4 on 4 df, p=0 Wald test = 111.1 on 4 df, p=0 Score (logrank) test = 179.9 on 4 df, p=0> with(redo, table(LOEStatusfull, Period,ExpTemp)), , ExpTemp = FIVE Period LOEStatusfull A B C D 0 42 0 35 0 1 4 0 11 0 , , ExpTemp = FOUR Period LOEStatusfull A B C D 0 0 0 0 2 1 0 46 0 44 As best as I can tell, none of my variables are collinear. Are there any other suggestions of how to deal with this error, or any more information I can provide to help understand why I would be getting this? Thank you for your time and your help, Katie On Sat, Jan 12, 2013 at 4:54 PM, Bert Gunter <gunter.berton at gene.com> wrote:> Katie: > > You need to get local statistical help. What you are doing makes no > sense. See inline below. > > -- Bert > > On Sat, Jan 12, 2013 at 1:03 PM, David Winsemius <dwinsemius at comcast.net> wrote: >> >> On Jan 11, 2013, at 5:35 PM, Katie Anweiler wrote: >> >>> Hello all, >>> I am trying to understand how to specify nested factors when using >>> coxph(), and if it is appropriate to nest these factors in my >>> situation. >>> In the simplest form, I am testing two different temperatures, with >>> each temperature being performed twice in different experimental >>> periods (e.g. Temp5 performed in Period A and C, Temp4 performed in >>> Period B and D) > > Period is confounded with temperature. That is the source of the > singularity. in the message received below. > > You can estimate the C-A and the D-B differences. > As I said, get statistical help. These are not R questions. > > -- Bert > > >>> I am trying to see if survival time is affected by the treatment >>> temperature. To do this I am using temperature and experimental >>> period nested within temperature as factors. >>> >>>> LOEtempmod.5days=coxph(LOE.stable.5days~Temp+Temp/Period,data=goodstable) >>> >>> Warning message: >>> In coxph(LOE.stable.5days ~ Temp + Temp/Period, : >>> X matrix deemed to be singular; variable 2 5 6 7 >>> >>> 1. Is this an appropriate way of nesting? >> >> >> Have you looked at the coxme package? >> http://cran.r-project.org/web/packages/coxme/index.html >> >>> 2. Can this error message be ignored? >> >> >> Sometimes R packages correctly drop variables that are exactly collinear: >> other times the correct solution is not clear. I would think the answer in >> this case would be "no", but do not have a lot to go on at this point. >> >> -- >> >> David Winsemius, MD >> Alameda, CA, USA >> >> ______________________________________________ >> R-help at r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-help >> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html >> and provide commented, minimal, self-contained, reproducible code. > > > > -- > > Bert Gunter > Genentech Nonclinical Biostatistics > > Internal Contact Info: > Phone: 467-7374 > Website: > http://pharmadevelopment.roche.com/index/pdb/pdb-functional-groups/pdb-biostatistics/pdb-ncb-home.htm
In a Cox model, the baseline hazard takes the place of an intercept.
Try the following:
dummy <- rnorm(nrow(goodexp))
testfit <- lm(dummy ~ ExpTemp + Stability + Period, data=goodexp, x=T)
Now look at testfit$x, which is the design matrix for the linear model with an
intercept.
Examination of it will first of all clarify for you exactly what dummy variable
coding was
used for the model. If this X matrix is singular, the coxph fit will be
singular in
exactly the same way and at the same place. If you regress the last column of
testfit$x
on the others you should get a perfect fit, given the output below. Other than
the
intercept the X matrix within coxph is the same as that for lm; the
"intercept" is still
there in the form of a baseline hazard, it just cannot be summarized as a single
coefficient. (All R modeling functions use the same internal routine to
generate the
design matrix.)
My guess is that the row sums of testfit$x are constant, but that's just
a guess.
Terry Therneau
PS -- use the spacebar more when showing an example. It makes it a lot easier
for the
rest of us to read.
On 01/24/2013 05:00 AM, r-help-request@r-project.org
wrote:> Thank you for the suggestions.
>
> Just to clarify, my first question was more on what actual coding I
> should be using to indicate a nested variable when using the coxph()
> function. I asked this after consulting several times with a local
> statistician, but unfortunately neither of us are very familiar with
> R.
>
> After further consultation, I have changed the design to a 2*2 design
> (2 levels of ExpTemp and Stability each) with blocking (Period). I am
> still getting the "x matrix deemed to be singular" error.
>
>> > LOEmod3alt=coxph(LOE.fit~ExpTemp+Stability+Period,data=goodexp)
> Warning message:
> In coxph(LOE.fit ~ ExpTemp + Stability + Period, data = goodexp) :
> X matrix deemed to be singular; variable 5
>> > summary(LOEmod3alt)
> Call:
> coxph(formula = LOE.fit ~ ExpTemp + Stability + Period, data = goodexp)
>
> n= 184, number of events= 105
>
> coef exp(coef) se(coef) z Pr(>|z|)
> ExpTemp -3.17825 0.04166 0.53105 -5.985 2.17e-09 ***
> StabilityStatic -0.84129 0.43115 0.20470 -4.110 3.96e-05 ***
> PeriodB 1.06794 2.90937 0.22859 4.672 2.98e-06 ***
> PeriodC 1.23853 3.45054 0.58457 2.119 0.0341 *
> PeriodD NA NA 0.00000 NA NA
> ---
> Signif. codes: 0 ?***? 0.001 ?**? 0.01 ?*? 0.05 ?.? 0.1 ? ? 1
>
> exp(coef) exp(-coef) lower .95 upper .95
> ExpTemp 0.04166 24.0047 0.01471 0.118
> StabilityStatic 0.43115 2.3194 0.28866 0.644
> PeriodB 2.90937 0.3437 1.85877 4.554
> PeriodC 3.45054 0.2898 1.09723 10.851
> PeriodD NA NA NA NA
>
> Concordance= 0.833 (se = 0.03 )
> Rsquare= 0.591 (max possible= 0.995 )
> Likelihood ratio test= 164.4 on 4 df, p=0
> Wald test = 111.1 on 4 df, p=0
> Score (logrank) test = 179.9 on 4 df, p=0
>
>> > with(redo, table(LOEStatusfull, Period,ExpTemp))
> , , ExpTemp = FIVE
>
> Period
> LOEStatusfull A B C D
> 0 42 0 35 0
> 1 4 0 11 0
>
> , , ExpTemp = FOUR
>
> Period
> LOEStatusfull A B C D
> 0 0 0 0 2
> 1 0 46 0 44
>
> As best as I can tell, none of my variables are collinear. Are there
> any other suggestions of how to deal with this error, or any more
> information I can provide to help understand why I would be getting
> this?
>
> Thank you for your time and your help,
>
> Katie
>
> O
[[alternative HTML version deleted]]