David Winsemius
2013-Sep-28 15:04 UTC
[R] What is a "good fit" Brier score and Harrel's C Index
On Sep 28, 2013, at 8:14 AM, E Joffe wrote:> > Hi all, > > I am evaluating survival models using Brier score ("peperr") and > Harrel's > C-Index ("Hmisc").It's spelled 'Harrell'.> I am wondering: > > 1. What would be considered a "good fit" according to these scores > (like the > heuristic levels we have for R square in linear regressions) ? > > 2. Are there any papers to cite on the matter (I couldn't find any) ?Frank Harrell's excellent text "Regression Modeling Strategies" has an extensive discussion of "goodness of fit" and the principles of model comparison. It's both too involved as well as off-topic for Rhelp. The other text to consult is Steyerberg's "Clinical Prediction Models".> > 3. Is there any paper to cite that discusses the limitation of using > traditional reporting for model fit in survival analysis as opposed > to these > measures ?I predict that the RMS bibliography would be an excellent place to start your search. Despite getting his name attached to what he calls the 'c-index', I don't think one could call Frank Harrell a proponent of that measure or any of the "competitors". It's really just a dressed up/transformed AUC. The message I have taken from reading his book and listening to presentations is that one should apply biologic tests of sensibility as well as careful investigation of the functional relationships between candidate predictors and the outcomes of interest. He speaks very disparagingly about automatic procedures. -- David Winsemius, MD Alameda, CA, USA
Hi all, I am evaluating survival models using Brier score ("peperr") and Harrel's C-Index ("Hmisc"). I am wondering: 1. What would be considered a "good fit" according to these scores (like the heuristic levels we have for R square in linear regressions) ? 2. Are there any papers to cite on the matter (I couldn't find any) ? 3. Is there any paper to cite that discusses the limitation of using traditional reporting for model fit in survival analysis as opposed to these measures ? Thanks, Erel