R help - Oct 2006 - Censored Brier Score and Royston/Sauerbrei's D

System: R 2.3.1 on a Windows XP computer. 

I am validating several cancer prognostic models that have been
published with a large independent dataset.  Some of the models report a
probability of survival at a specified timepoint, usually at 5 and 10
years.  Others report only the linear predictor of the Cox model.

I have used Harrell's c index for censored data (rcorr.cens) as a
measure of discrimination and have constructed smoothed calibration
plots.  I would like to include some measures of overall model
performance, such as the censored Brier score and Royston/Sauerbrei's D
statistic (Stat Med 2004). With this in mind, I have 3 questions:

1) Can the "sbrier" function of the "ipred" library be used
to calculate
the censored Brier score for a specific time point given known
predictions for that timepoint?  

library(ipred)

data <- read.csv(file='c:\\....

time    <- data$time  	# The time in years from diagnosis of cancer to
death
status <- data$status 	# The status at last follow-up: 1=dead, 0=alive
pred    <- data$pred	# The predicted probability of surviving 5 years
after cancer from external Cox model A
linp     <- data$linp	# The linear predictor of external Cox model B
predicting survival after cancer 

s <- Surv(time, status)
test <- sbrier(s, pred, btime=5)

# I get this error message that I can't seem to solve
Error in Surv(time, 1 - cens) : Time variable is not numeric
In addition: Warning message:
is.na() applied to non-(list or vector) in: is.na(time)

2) Can the linear predictor (linp) be used in sbrier in the same way as
a probability might?

3) Has anyone implemented Royston/Sauerbrei's D?

Brant Inman
Mayo Clinic


	[[alternative HTML version deleted]]

On Fri, 27 Oct 2006, Inman, Brant A.   M.D. wrote:
> System: R 2.3.1 on a Windows XP computer.
>
> I am validating several cancer prognostic models that have been
> published with a large independent dataset.  Some of the models report a
> probability of survival at a specified timepoint, usually at 5 and 10
> years.  Others report only the linear predictor of the Cox model.
>
> I have used Harrell's c index for censored data (rcorr.cens) as a
> measure of discrimination and have constructed smoothed calibration
> plots.  I would like to include some measures of overall model
> performance, such as the censored Brier score and Royston/Sauerbrei's D
> statistic (Stat Med 2004). With this in mind, I have 3 questions:
>
please make sure to cc maintainers of contributed packages -- you can't 
assume that everybody follows r-help on a regular basis.
> 1) Can the "sbrier" function of the "ipred" library be
used to calculate
                                               ^^^^^^^ <- package
just to make Martin happy :-)
> the censored Brier score for a specific time point given known
> predictions for that timepoint?
>
> library(ipred)
>
> data <- read.csv(file='c:\\....
>
> time    <- data$time  	# The time in years from diagnosis of cancer to
> death
> status <- data$status 	# The status at last follow-up: 1=dead, 0=alive
> pred    <- data$pred	# The predicted probability of surviving 5 years
> after cancer from external Cox model A
> linp     <- data$linp	# The linear predictor of external Cox model B
> predicting survival after cancer
>
> s <- Surv(time, status)
> test <- sbrier(s, pred, btime=5)
>
should work and the manual page as an example for this situation.
> # I get this error message that I can't seem to solve
> Error in Surv(time, 1 - cens) : Time variable is not numeric
> In addition: Warning message:
> is.na() applied to non-(list or vector) in: is.na(time)
what does summary(time) say? If the problem persists please send me a 
reproducible example I can debug (off-list).
>
> 2) Can the linear predictor (linp) be used in sbrier in the same way as
> a probability might?
>
no. But have a look at `survival:::survfit.coxph' for how to compute an 
estimate of the survival function based on the a fitted Cox model.

Best wishes,

Torsten

> 3) Has anyone implemented Royston/Sauerbrei's D?
>
> Brant Inman
> Mayo Clinic
>
>
> 	[[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
>