Hi, I want to fit multiple cox models using the coxph() function. To do this, I use a for-loop and save the relevant results in a separate matrix. In the example below, only two models are fitted (my actual matrix has many more columns), one gives a warning message, while the other does not. Right now, I see all the warning message(s) after the for-loop is completed but have no idea which model gave the warning message. Is there a way in which the warning message can be captured and saved (i.e. as a binary variable, having value 1 if there was a warning message and 0 otherwise)? I can't possibly fit the models one by one (and see if they give a warning message) as I have many of them to fit.> library(survival)Loading required package: splines> time= c(4,3,1,1,2,2,3,3,2) > status=c(1,0,0,0,1,1,1,1,1) > TIME=Surv(time,status) > x= cbind(c(0,2,1,1,0,0,0,2,0),c(0,2,1,1,0,0,0,0,0)) > > results=matrix(NA,ncol=3,nrow=ncol(x)) > colnames(results)=c("coef","se","p") > > for(i in 1:ncol(x)){+ fit=summary(coxph(TIME~x[,i])) + results[i,1]=fit$coef[1] + results[i,2]=fit$coef[3] + results[i,3]=fit$coef[5] + rm(fit) + } Warning message: Loglik converged before variable 1 ; beta may be infinite. in: fitter(X, Y, strats, offset, init, control, weights = weights,> > resultscoef se p [1,] -0.5117033 5.647385e-01 0.36 [2,] -10.2256937 1.146168e+04 1.00> > #To see which model gave the warning message > coxph(TIME~x[,1])Call: coxph(formula = TIME ~ x[, 1]) coef exp(coef) se(coef) z p x[, 1] -0.512 0.6 0.565 -0.906 0.36 Likelihood ratio test=0.97 on 1 df, p=0.324 n= 9> coxph(TIME~x[,2])Call: coxph(formula = TIME ~ x[, 2]) coef exp(coef) se(coef) z p x[, 2] -10.2 3.62e-05 11462 -0.000892 1 Likelihood ratio test=2.51 on 1 df, p=0.113 n= 9 Warning message: Loglik converged before variable 1 ; beta may be infinite. in: fitter(X, Y, strats, offset, init, control, weights = weights, Thank you, Cindy Lin
One way is to turn the 'warnings' into 'errors' and then trap the error:> library(survival) > > time= c(4,3,1,1,2,2,3,3,2) > status=c(1,0,0,0,1,1,1,1,1) > TIME=Surv(time,status) > x= cbind(c(0,2,1,1,0,0,0,2,0),c(0,2,1,1,0,0,0,0,0)) > > results=matrix(NA,ncol=3,nrow=ncol(x)) > colnames(results)=c("coef","se","p") > > old.warn <- options(warn=2) > for(i in 1:ncol(x)){+ + aa <- try(fit <- summary(coxph(TIME~x[,i]))) + if (class(aa) == "try-error"){ + print(paste("i =", i, "had error")) + next # skip iteration + } + + results[i,1]=fit$coef[1] + results[i,2]=fit$coef[3] + results[i,3]=fit$coef[5] + rm(fit) + } Error in fitter(X, Y, strats, offset, init, control, weights = weights, : (converted from warning) Loglik converged before variable 1 ; beta may be infinite. [1] "i = 2 had error"> options(old.warn) > >On Dec 17, 2007 10:16 AM, xinyi lin <x1lin at ucsd.edu> wrote:> Hi, > > I want to fit multiple cox models using the coxph() function. To do > this, I use a for-loop and save the relevant results in a separate > matrix. In the example below, only two models are fitted (my actual > matrix has many more columns), one gives a warning message, while the > other does not. Right now, I see all the warning message(s) after the > for-loop is completed but have no idea which model gave the warning > message. Is there a way in which the warning message can be captured > and saved (i.e. as a binary variable, having value 1 if there was a > warning message and 0 otherwise)? I can't possibly fit the models one > by one (and see if they give a warning message) as I have many of them > to fit. > > > > library(survival) > Loading required package: splines > > time= c(4,3,1,1,2,2,3,3,2) > > status=c(1,0,0,0,1,1,1,1,1) > > TIME=Surv(time,status) > > x= cbind(c(0,2,1,1,0,0,0,2,0),c(0,2,1,1,0,0,0,0,0)) > > > > results=matrix(NA,ncol=3,nrow=ncol(x)) > > colnames(results)=c("coef","se","p") > > > > for(i in 1:ncol(x)){ > + fit=summary(coxph(TIME~x[,i])) > + results[i,1]=fit$coef[1] > + results[i,2]=fit$coef[3] > + results[i,3]=fit$coef[5] > + rm(fit) > + } > Warning message: > Loglik converged before variable 1 ; beta may be infinite. in: > fitter(X, Y, strats, offset, init, control, weights = weights, > > > > results > coef se p > [1,] -0.5117033 5.647385e-01 0.36 > [2,] -10.2256937 1.146168e+04 1.00 > > > > #To see which model gave the warning message > > coxph(TIME~x[,1]) > Call: > coxph(formula = TIME ~ x[, 1]) > > > coef exp(coef) se(coef) z p > x[, 1] -0.512 0.6 0.565 -0.906 0.36 > > Likelihood ratio test=0.97 on 1 df, p=0.324 n= 9 > > coxph(TIME~x[,2]) > Call: > coxph(formula = TIME ~ x[, 2]) > > > coef exp(coef) se(coef) z p > x[, 2] -10.2 3.62e-05 11462 -0.000892 1 > > Likelihood ratio test=2.51 on 1 df, p=0.113 n= 9 > Warning message: > Loglik converged before variable 1 ; beta may be infinite. in: > fitter(X, Y, strats, offset, init, control, weights = weights, > > > Thank you, > Cindy Lin > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. >-- Jim Holtman Cincinnati, OH +1 513 646 9390 What is the problem you are trying to solve?
Xinyi Li asked how to keep track of which coxph models, called from within a
loop, were responsible for warning messges. One solution is to modify the
coxph code so that those models are marked in the return coxph object. Below
is a set of changes to the final 40 lines of coxph.fit, that will cause the
component "infinite.warn" to be added to the result whenever a warning
was
generated; it will be a vector of T/F showing which component(s) of the
coefficient vector generated the warning.
Change the code for coxph.fit.s, then do
> source('coxph.fit.revised.s') #or whatever you called it
> coxph <- source('coxph.s')
> coxph.wtest <- survival:::coxph.wtest
Line 2 causes you to have a local version of coxph, otherwise, due
to name spaces, the original version of coxph.fit will still be used. Line 3
is another consequence of name spaces. Then code such as
for(i in 1:ncol(x)){
fit=coxph(TIME~x[,i])
if (!is.null(fit$infinite.warn)) cat("Waring in fit", i,
"\n")
sfit <- summary(fit)
results[i,1]=sfit$coef[1]
results[i,2]=sfit$coef[3]
results[i,3]=sfit$coef[5]
}
Will report out the models with a warning.
Notes
1. The warning message is not completely reliable
2. Name spaces protect a package from accidental overrides, when a user or
some other package reuses a function name. With its hundreds of packages, they
are a necessity for R. But sometimes you really do want to override a
function; then they are a bit of a pain. Others with a better grasp of R
internals may be able to suggest a better override than I have done here.
Terry Therneau
-------------------------
infs <- abs(coxfit$u %*% var)
keep.infs <- F # new line
if (maxiter >1) {
if (coxfit$flag == 1000)
warning("Ran out of iterations and did not converge")
else {
infs <- ((infs > control$eps) &
infs > control$toler.inf*abs(coef))
if (any(infs))
warning(paste("Loglik converged before variable ",
paste((1:nvar)[infs],collapse=","),
"; beta may be infinite. "))
keep.infs <- T #new line
}
}
names(coef) <- dimnames(x)[[2]]
lp <- c(x %*% coef) + offset - sum(coef*coxfit$means)
score <- exp(lp[sorted])
coxres <- .C("coxmart", as.integer(n),
as.integer(method=='efron'),
stime,
sstat,
newstrat,
as.double(score),
as.double(weights),
resid=double(n))
resid <- double(n)
resid[sorted] <- coxres$resid
names(resid) <- rownames
coef[which.sing] <- NA
temp <- list(coefficients = coef, #modified line
var = var,
loglik = coxfit$loglik,
score = coxfit$sctest,
iter = coxfit$iter,
linear.predictors = as.vector(lp),
residuals = resid,
means = coxfit$means,
method='coxph')
if (keep.infs) temp$infinite.warn <- infs #new line
temp #new line
}
}