Packages pROC and ROCR both calculate/approximate the Area Under (Receiver Operator) Curve. However the results are different. I am computing a new variable as a predictor for a label. The new variable is a (non-linear) function of a set of input values, and I'm checking how different parameter settings contribute to prediction. All my settings are predictive, but some are better. The AUC i got with pROC was much lower then expected, so i tried ROCR. Here are some comparisons: AUC from pROC AUC from ROCR 0.49465 0.79311 0.49465 0.79349 0.49701 0.79446 0.49701 0.79764 When i draw the ROC (with pROC) i get the curve i expect. But why is the AUC according to pROC so different? Ivana [[alternative HTML version deleted]]
A reproducible example sent to the package maintainer(s) might yield results. Max On Wed, Dec 19, 2012 at 7:47 AM, Ivana Cace <I.Cace@ati-a.nl> wrote:> Packages pROC and ROCR both calculate/approximate the Area Under (Receiver > Operator) Curve. However the results are different. > > I am computing a new variable as a predictor for a label. The new variable > is a (non-linear) function of a set of input values, and I'm checking how > different parameter settings contribute to prediction. All my settings are > predictive, but some are better. > > The AUC i got with pROC was much lower then expected, so i tried ROCR. > Here are some comparisons: > AUC from pROC AUC from ROCR > 0.49465 0.79311 > 0.49465 0.79349 > 0.49701 0.79446 > 0.49701 0.79764 > > When i draw the ROC (with pROC) i get the curve i expect. But why is the > AUC according to pROC so different? > > Ivana > > > > > [[alternative HTML version deleted]] > > ______________________________________________ > R-help@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. >-- Max [[alternative HTML version deleted]]
On Dec 19, 2012, at 4:47 AM, Ivana Cace wrote:> Packages pROC and ROCR both calculate/approximate the Area Under (Receiver Operator) Curve. However the results are different. > > I am computing a new variable as a predictor for a label. The new variable is a (non-linear) function of a set of input values, and I'm checking how different parameter settings contribute to prediction. All my settings are predictive, but some are better. > > The AUC i got with pROC was much lower then expected, so i tried ROCR. Here are some comparisons: > AUC from pROC AUC from ROCR > 0.49465 0.79311 > 0.49465 0.79349 > 0.49701 0.79446 > 0.49701 0.79764 > > When i draw the ROC (with pROC) i get the curve i expect. But why is the AUC according to pROC so different?Why are you sending this to the Rhelp mailing list? You should instead be reporting it to the package maintainers.> maintainer('ROCR')[1] "Tobias Sing <tobias.sing at mpi-sb.mpg.de>"> maintainer('pROC')[1] "Xavier Robin <Xavier.Robin at unige.ch>" -- David Winsemius Alameda, CA, USA