R help - Sep 2013 - What is a "good fit" Brier score and Harrel's C Index

On Sep 28, 2013, at 8:14 AM, E Joffe wrote:
>
> Hi all,
>
> I am evaluating survival models using Brier score ("peperr") and
> Harrel's
> C-Index ("Hmisc").
It's spelled 'Harrell'.
> I am wondering:
>
> 1. What would be considered a "good fit" according to these
scores
> (like the
> heuristic levels we have for R square in linear regressions) ?
>
> 2. Are there any papers to cite on the matter (I couldn't find any) ?
Frank Harrell's excellent text "Regression Modeling Strategies"
has an
extensive discussion of "goodness of fit" and the principles of model
comparison. It's both too involved as well as off-topic for Rhelp. The  
other text to consult is Steyerberg's "Clinical Prediction
Models".
>
> 3. Is there any paper to cite that discusses the limitation of using
> traditional reporting for model fit in survival analysis as opposed  
> to these
> measures  ?
I predict that the RMS bibliography would be an excellent place to  
start your search.

Despite getting his name attached to what he calls the 'c-index', I  
don't think one could call Frank Harrell a proponent of that measure  
or any of the "competitors". It's really just a dressed
up/transformed
AUC. The message I have taken from reading his book and listening to  
presentations is that one should apply biologic tests of sensibility  
as well as careful investigation of the functional relationships  
between candidate predictors and the outcomes of interest. He speaks  
very disparagingly about automatic procedures.

-- 

David Winsemius, MD
Alameda, CA, USA

Hi all, 

I am evaluating survival models using Brier score ("peperr") and
Harrel's
C-Index ("Hmisc").
I am wondering:

1. What would be considered a "good fit" according to these scores
(like the
heuristic levels we have for R square in linear regressions) ?

2. Are there any papers to cite on the matter (I couldn't find any) ?

3. Is there any paper to cite that discusses the limitation of using
traditional reporting for model fit in survival analysis as opposed to these
measures  ?

Thanks,
Erel