search for: brier

Hi, I have probability estimates from two predictive models. I have these estimates and also a binary outcome for a validation data set not used in calibrating either model. I would like to calculate the Brier score for both models on this binary outcome and test the hypothesis that the Brier scores are equal from the two models. I have not been able to find an R function to do this, can someone point me to the appropriate library and function if one exists? Thanks, Seth -- View this message in contex...

Hello, I am using the val.prob function in the Design package. I understand how the Brier quadratic error score is calculated, but I do not know how the Brier score computed on the calibrated rather than raw predicted probabilities (B cal) is calculated. My question is: how are the calibrated probabilities calculated? Any explanation of this, or references to explanations of this, wou...

...ars. Others report only the linear predictor of the Cox model. I have used Harrell's c index for censored data (rcorr.cens) as a measure of discrimination and have constructed smoothed calibration plots. I would like to include some measures of overall model performance, such as the censored Brier score and Royston/Sauerbrei's D statistic (Stat Med 2004). With this in mind, I have 3 questions: 1) Can the "sbrier" function of the "ipred" library be used to calculate the censored Brier score for a specific time point given known predictions for that timepoint? libra...

Dear all, I would like to obtain the Brier score prediction error at different times t for an extended Cox model. Previously I have used the 'pec' function (pec{pec}) to obtain prediction error curves for standard Cox PH models but now I have data in the counting process format (I have a covariate with a time-varying effect) and it...

Hello I've decided to try and distill an earlier rather ill focused question to try and elicit a response. Any help is greatly appreciated. Why does mod.cox not work with sbrier whilst mod.km does? Can I make it work? > data(DLBCL) > DLBCL.surv<-Surv(DLBCL$time,DLBCL$cens) > > mod.km<-survfit(DLBCL.surv) > mod.cox<-survfit(coxph(DLBCL.surv~IPI, data=DLBCL)) > > > sbrier(DLBCL.surv, mod.km) integrated Brier score 0.207645...

...ur opinion ? More precisely, what problems are there in MS Office apps emulation ? Is there any kind of technical synthesis available on the subject ? Apart from that, is there any idea of the predictible date for a professional usable release ? Thank you in advance for help. Yours. Philippe BRIERE

...e response variable is classed as 1,2,3,4,5,6, this relates to threat level of the species ( on the IUCN rating). Previously i have combined levels 1 and 2 to form = non threatened and then combined 3-6 to form threatened, and run a binary model. I have tested the performance of this based on the brier score (0.198) and the AUC or C value (0.75). I also partitioned the data set into training and test data and used the predict function to get a predicted probability for the newdata. When visualising the results with a cutoff value calculated with epi, roughly 75% of the time the prediction was cor...

from samba 3.0.9 to 3.0.13 I can not access my shares from a NT4 box as usual before upgrading. I can access samba shares from linux via mount //server/testuser /mnt -o username=testuser,password=testpass as before. Even the newest version (3.0.14a) from samba.org doesnt't help. What has happened to samba ??? Any hints ?? Thanks for your thoughts, G?tz.

Dear List, I am looking to run a host of models (60) with three methods - lmer,glm and lrm. Is there a way to output the key stats into a table that i can copy to excel? I.e for lmer i would want AIC,BIC etc for lrm i would want Brier score, r2, c-value etc At present i am running the models from a script and then copying across the values into a excel spreadsheet however this is time consuming and i imagine their must be a trick? Thanks Chris

dear:teacher i have a problem which about the polr()(package "MASS"), if the response must have 3 or more levels? and how to fit the polr() to 2 levels? thank you. turly yours [[alternative HTML version deleted]]

...re some details from my workspace: Reformat_dataSet: 368 obs. Of 356 variables print (glmnet.cox) ----> 354 df, p=1 n= 368, number of events= 288 (354 df = 354 variables + time and status => 356 variables) Here is the pec function and the error: pec.f <- as.formula(Hist(time,status) ~ 1) brierGlmnet <- pec(list(glmnet.cox), data = reformat_Dataset, splitMethod="BootCV", B=50, formula = pec.f) > Error in predictSurvProb.coxph(object = structure(list(coefficients = structure(c(-4.27787223119601, : Prediction matrix has wrong dimensions: 368 rows and 318 columns....

...NDWI <- lrm(m.arson ~ rcs(NDWI,knots) I obtain the following result: Logistic Regression Model lrm(formula = m.arson ~ rcs(NDWI, knots)) Frequencies of Responses 0 1 666 35 Obs Max Deriv Model L.R. d.f. P C Dxy Gamma Tau-a R2 Brier 701 5e-07 34.49 4 0 0.777 0.553 0.563 0.053 0.147 0.045 Coef S.E. Wald Z P Intercept -4.627 3.188 -1.45 0.1467 NDWI 5.333 20.724 0.26 0.7969 NDWI' 6.832 74.201 0.09 0.9266 NDWI'...

...R2 0.088 C 0.607 0 46 d.f. 1 g 0.488 Dxy 0.215 1 36 Pr(> chi2) 0.0182 gr 1.629 gamma 0.589 max |deriv| 9e-11 gp 0.107 tau-a 0.107 Brier 0.231 Coef S.E. Wald Z Pr(>|Z|) Intercept -1.3218 0.5627 -2.35 0.0188 X16=1 1.3535 0.6166 2.20 0.0282 ----------------------------------------------------------- Analyzing the goodness of fit: ----------------------------------------------------------- > resid(mo...

...ormula = y ~ x1 + x2 + x3, data = dat, x = T, y = T) Frequencies of Responses 1 2 50 50 Obs Max Deriv Model L.R. d.f. P C Dxy Gamma 100 1e-08 3.61 3 0.3066 0.594 0.187 0.188 Tau-a R2 Brier 0.095 0.047 0.241 Coef S.E. Wald Z P Intercept -0.05224 0.2071 -0.25 0.8009 x1 -0.30699 0.1952 -1.57 0.1159 x2 0.08093 0.2171 0.37 0.7093 x3 0.21450 0.2247 0.95 0.3398

Package: logcheck Version: 1.2.39 Severity: normal Since I've upgraded my servers to sarge, I'm getting mail every hour for stuff that was duly included in /etc/logcheck/logcheck.ignore. Turns out that sarge's version no longer reads that file. If this was a conscious decision, then there should be some warning about this when upgrading (via debconf of NEWS.Debian). Also, the

...nce: > model.lrm24$stats Obs Max Deriv Model L.R. d.f. P 3.610000e+02 5.352815e-02 1.251547e+02 4.800000e+01 8.340322e-09 C Dxy Gamma Tau-a R2 8.795642e-01 7.591284e-01 7.631040e-01 2.578793e-01 4.522090e-01 Brier g gr gp 1.113480e-01 3.489163e+00 3.275851e+01 2.494908e-01 So it still seems it was able to run, at least to some extent, despite it's protests. Any ideas what is going on? Why does the model run if it is "unable" to be fit? That seems contradictory...

...ata$v1, trainlogdata$v2), y = trainlogdata$label) Frequencies of Responses 0 1 20988 6684 Obs Max Deriv Model L.R. d.f. P C Dxy 27672 8e-13 3876.12 2 0 0.744 0.488 Gamma Tau-a R2 Brier 0.49 0.179 0.195 0.157 Coef S.E. Wald Z P Intercept -2.890 0.04318 -66.92 0 x[1] 5.864 0.23341 25.12 0 x[2] 4.479 0.17256 25.95 0 > predictions <- predict(fmodel, trainlogdata, type="fitted"); Error in getOldDesign(fit) :

...0.763 0 128 d.f. 1 g 1.209 Dxy 0.526 1 22 Pr(> chi2) <0.0001 gr 3.350 gamma 0.528 max |deriv| 1e-11 gp 0.129 tau-a 0.132 Brier 0.111 Coef S.E. Wald Z Pr(>|Z|) Intercept -5.0059 0.9813 -5.10 <0.0001 x 0.5647 0.1525 3.70 0.0002 As you can see, the odds ratio for x is exp(0.5647)=1.75892. But if I run the following using summary(): > summary(fit) E...

...nection, so the client doesn't appear to be broadcasting. Yet it's not closing the connection either. I'm attaching ices2's config file. Any help or pointers would be appreciated; otherwise I'll have to sprinkle some fprintfs everywhere and hope for the best. :( -- Frederic Briere <*> fbriere@fbriere.net => <fbriere@abacom.com> IS NO MORE: <http://www.abacomsucks.com> <= -------------- next part -------------- A non-text attachment was scrubbed... Name: ices2.xml Type: text/xml Size: 2145 bytes Desc: not available Url : http://lists.xiph.or...

...xy 0.386 1 116 Pr(> chi2) <0.0001 gr 2.400 gamma 0.408 max |deriv| 2e-09 gp 0.183 tau-a 0.180 Brier 0.207 Coef S.E. Wald Z Pr(>|Z|) Intercept -2.3566 0.3819 -6.17 <0.0001 Complications 1.6807 0.6005...