search for: auc

Hi, I was lately debugging parts of my 'colAUC' function in caTools package, and in a process looked into other packages for calculating Areas Under ROC Curves (AUC). To my surprise I found at least 6 other functions: * wilcox.test * AUC from ROC package, * performance from ROCR package, * auROC from limma package, * ROC from...

Hi I am trying to perform a bootstrap estimate of classification accuracy of a logistic regression using the 'Daim' package in r using the code at the bottom of this post, this all works great and I get the .632+ misclassification accuracy, specificity, sensitivity, AUC etc etc but what I would like is to access the list of AUC for each of the bootstrap samples as I need calculate the 95% confidence intervals for the AUC of the ROC curve for this data using this model. I was hoping to get this from the 2.5% and 97.5% quartiles of the AUCs of the 10,000 bootstrap s...

Dear all, I am using the ROCR library to compute the AUC and also the Hmisc library to compute the C-index of a predictor and a group variable. The results of AUC and C-index are similar and give a value of about 0.57. The Wilcoxon p-value is <0.001! Why the AUC is showing small value and the p-value is high significant? The AUC is based on Wilcoxon c...

Hi, I would like to compare differences in AUC from 2 different models, glm and gam for predicting presence / absence. I know that in theory the model with a higher AUC is better, but what I am interested in is if statistically the increase in AUC from the glm model to the gam model is significant. I also read quite extensive discussions on the...

...help is much appreciated. The continuous information provided in this forum is fabulous as are the different R packages available. Rene # Simulated simplified data Subj <- rep(1:4, each=6) Analyte <- rep(c(rep("RBV",3),rep("TBV",3)),4) Dose <- rep(c(200,400,600),8) AUC <- rnorm(24, c(40,80,120,4,8,12), c(8,16,24,0.8,0.16,0.24)) # The real dataset may have NAs in it df <- data.frame(Subj, Analyte, Dose, AUC) myStats <- function(x) { count <- function(x) length(na.omit(x)) pCV <- function(x) sd(x,T) / mean(x,T) * 100 c( n = count(...

Dear R-philes, I am plotting ROC curves for several cross-validation runs of a classifier (using the function below). In addition to the average AUC, I am interested in obtaining a confidence interval for the average AUC. Is there a straightforward way to do this via the ROCR package? plot_roc_curve <- function(roc.dat, plt.title) { #print(str(vowel.ROC)) pred <- prediction(roc.dat$predictions, roc.dat$labels) perf <- performa...

Hi, there. Suppose I already have sensitivities and specificities. What is the quick R-function to calculate AUC for the ROC plot? There seem to be many R functions to calculate AUC. Thanks. Yulei [[alternative HTML version deleted]]

Dear List, I am trying to assess the prediction accuracy of an ordinal model fit with LRM in the Design package. I used predict.lrm to predict on an independent dataset and am now attempting to assess the accuracy of these predictions. >From what I have read, the AUC is good for this because it is threshold independent. I obtained the AUC for the fit model output from the c score (c = 0.78). For the predicted values and independent data, for each level of the response I used the ROCR functions to get the AUC (i.e., probability y >= class1, y >= class2, y...

I am doing a simulation with a relatively large data set (20,000 observations) for which I want to calculate the area under the Receiver Operator Curve (AUC) for many parameter combinations. I am using the ROC library and the following commands to generate each AUC: rocobj=rocdemo.sca(truth = ymis, data = model$fitted.values, rule = dxrule.sca) #generation of observed ROC object aucobj=AUC(rocobj) #pulling out just the observed AUC - trapezoidal...

Dear R-experts: I am using a function called AUC whose arguments are data, time, id, and dv. data is the name of the dataframe, time is the independent variable column name, id is the subject id and dv is the dependent variable. The function computes area under the curve by trapezoidal rule, for each subject id. I would like to embed t...

Dear all, I was wondering if someone can help me. I am learning SVM for classification in my research with kernlab package. I want to know about classification performance using Area Under Curve (AUC). I know ROCR package can do this job but I found all example in ROCR package have include prediction, for example, ROCR.hiv {ROCR}. My problem is how to produce prediction in SVM and to find AUC. Here is a simple example: library(MASS) library(kernlab) library(ROCR) pimamodel <- ksvm(type...

...wever the results are different. I am computing a new variable as a predictor for a label. The new variable is a (non-linear) function of a set of input values, and I'm checking how different parameter settings contribute to prediction. All my settings are predictive, but some are better. The AUC i got with pROC was much lower then expected, so i tried ROCR. Here are some comparisons: AUC from pROC AUC from ROCR 0.49465 0.79311 0.49465 0.79349 0.49701 0.79446 0.49701 0.79764 When i draw the ROC (wit...

Dear all I am trying to compare the performances of several methods using the AUC0.1 and not the whole AUC. (meaning I wanted to compare to AUC's whose x axis only goes to 0.1 not 1) I came to know about the Mcneil Hanley test from Bernardo Rangel Tura and I referred to the original paper for the calculation of "r" which is an argument of the function cROC. I can...

...rnorm(n,0,tau) model<-glm(z ~ xerr, family=binomial(logit)) int<-coef(model)[1] slope<-coef(model)[2] pred<-predict(model) result<-ifelse(pred>.5,1,0) accuracy<-length(which(result==z))/length(z) accuracy rocpreds<-prediction(result,z) auc<-performance(rocpreds,"auc")@y.values sentiv<-performance(rocpreds,"sens")@y.values sentiv<-slot(fp,"y.values")[[1]] sentiv<-sentiv[2] sentiv specs<-performance(rocpreds,"spec")@y.values specs<-slot(fp2,"y.values...

Guys, I used Random Forest with a couple of data sets I had to predict for binary response. In all the cases, the AUC of the training set is coming to be 1. Is this always the case with random forests? Can someone please clarify this? I have given a simple example, first using logistic regression and then using random forests to explain the problem. AUC of the random forest is coming out to be 1. data(iris) iri...

plot(svm.auc, col=2, main="ROC curves comparing classification performance\n of six machine learning models") legend(0.5, 0.6, c(ns, nb, nr, nt, nl,ne), 2:6, 9) # Draw a legend. plot(bo.auc, col=3, add=T) # add=TRUE draws on the existing chart plot(rf.auc, col=4, add=T) plot(tree.auc, col=5, add=T) p...

Hello Is there an easy way, i.e. a function in a package, to calculate the area under the curve (AUC) for drug serum levels? Thanks for any advice -- Armin Goralczyk, M.D. -- Universit?tsmedizin G?ttingen Abteilung Allgemein- und Viszeralchirurgie Rudolf-Koch-Str. 40 39099 G?ttingen -- Dept. of General Surgery University of G?ttingen G?ttingen, Germany -- http://www.chirurgie-goettingen.de

Hallo! Does anybody know a way to calculate the sample size for comparing AUC of ROC curves against 'by chance' with AUC=0.5 (and/or against anothe AUC)? Thanks! Karl

I could do this in various hacky ways, but what's the right way? I have a nice application of the by function, which does what I want. The output looks like this: > auc_stress lab.samples.stress$subid: 2 cortisol amylase 1 919.05 6834.8 --------------------------------------------------------------------------------------------------------------------------- lab.samples.stress$subid: 3 cortisol amylase 11 728.25 24422.05 etc. What I want is a data fr...

...dertake several steps (stepwise forward selection with LR-Test statistics, incorporating interaction effects etc.). The final prediction at the end however would be through a glm object (called fit.glm). Then, I think so, it would be no problem to calculate a Nagelkerke R squared measure and an AUC value (for example with the pROC package) following the script: BaseRate <- table(Data$Y[[1]])/sum(table(Data$Y)) L(0)=Likelihood(Null-Model)= (BaseRate*log(BaseRate)+(1-BaseRate)*log(1-BaseRate))*sum(table(Data$Y)) LIKM <- predict(fit.glm, type="response") L(M)=Likelihood(Fitted...