R help - Mar 2011 - function to compare Brier scores from two models?

Hi,

I have probability estimates from two predictive models.  I have these
estimates and also a binary outcome for a validation data set not used in
calibrating either model.  I would like to calculate the Brier score for
both models on this binary outcome and test the hypothesis that the Brier
scores are equal from the two models.  I have not been able to find an R
function to do this, can someone point me to the appropriate library and
function if one exists?  Thanks, Seth

--
View this message in context:
http://r.789695.n4.nabble.com/function-to-compare-Brier-scores-from-two-models-tp3409714p3409714.html
Sent from the R help mailing list archive at Nabble.com.

Hi:

Try

library(sos)       # install first if necessary
findFn('Brier score')

The first place I would look is Frank Harrell's rms package.

HTH,
Dennis

On Sun, Mar 27, 2011 at 1:50 PM, Seth <sjmyers@syr.edu> wrote:
> Hi,
>
> I have probability estimates from two predictive models.  I have these
> estimates and also a binary outcome for a validation data set not used in
> calibrating either model.  I would like to calculate the Brier score for
> both models on this binary outcome and test the hypothesis that the Brier
> scores are equal from the two models.  I have not been able to find an R
> function to do this, can someone point me to the appropriate library and
> function if one exists?  Thanks, Seth
>
> --
> View this message in context:
>
http://r.789695.n4.nabble.com/function-to-compare-Brier-scores-from-two-models-tp3409714p3409714.html
> Sent from the R help mailing list archive at Nabble.com.
>
> ______________________________________________
> R-help@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
	[[alternative HTML version deleted]]