similar to: AUC values from LRM and ROCR

Packages pROC and ROCR both calculate/approximate the Area Under (Receiver Operator) Curve. However the results are different. I am computing a new variable as a predictor for a label. The new variable is a (non-linear) function of a set of input values, and I'm checking how different parameter settings contribute to prediction. All my settings are predictive, but some are better. The AUC i

Dear R-philes, I am plotting ROC curves for several cross-validation runs of a classifier (using the function below). In addition to the average AUC, I am interested in obtaining a confidence interval for the average AUC. Is there a straightforward way to do this via the ROCR package? plot_roc_curve <- function(roc.dat, plt.title) { #print(str(vowel.ROC)) pred <-

Hi, I am quite new to R and I have two questions regarding ROCR. 1. I have tried to understand how to extract area-under-curve value by looking at the ROCR document and googling. Still I am not sure if I am doing the right thing. Here is my code, is "auc1" the auc value? " pred1 <- prediction(resp1,label1) perf1 <- performance(pred1,"tpr","fpr") plot(

Hello, I would like to use the ROCR package to draw ROC curves and compute AUC values. However, in the specific context of my application, the true positive rates and false positive rates are already provided by some upstream method. Of course, I can draw a ROC plot with the following command : plot(x=FPrate, y=TPrate, "o", xlab="false positive rate", ylab="true

Dear R-helper, I am working with ROCR of Tobias Sing et. al. to compare the performances of logistic and nnet models on a binary response. I had the performance plots, but I have problem finding out other performance statistics (eg. MSE/ASE, AUC). Any help on this? Thanks Ilham [[alternative HTML version deleted]]

Hi, I am using logistic regression model named lrm(Design) Rite now I was using Area Under Curve (AUC) for testing my model. But, now I have to calculate precision/recall of the model on test cases. For lrm, precision and recal would be simply defined with the help of 2 terms below: True Positive (TP) - Number of test cases where class 1 is given probability >= 0.5. False Negative (FP) -

Dear List, I am familier with binary models, however i am now trying to get predictions from a ordinal model and have a question. I have a data set made up of 12 categorical predictors, the response variable is classed as 1,2,3,4,5,6, this relates to threat level of the species ( on the IUCN rating). Previously i have combined levels 1 and 2 to form = non threatened and then combined 3-6 to

I am using logistic regression model (lrm) of package Design. Can some one please tell me how to calculate the average Area Under Curve (AUC) for n-fold cross-validation The help for lrm function says to do cross validation like this f <- lrm( cy ~ x1 + x2, x=TRUE, y=TRUE) val <- validate.lrm(f, method="cross", B=5) Now I dont know what to do with variable "val" to

Hi Does anyone know whether the rms package provides a confidence interval for the bootstrap-corrected Dxy or c-index? I have fitted a logistic model, and would like to obtain the 95% confidence interval of the bootstrap-corrected area under the ROC curve estimate. Thanks. [[alternative HTML version deleted]]

Dear all, I was wondering if someone can help me. I am learning SVM for classification in my research with kernlab package. I want to know about classification performance using Area Under Curve (AUC). I know ROCR package can do this job but I found all example in ROCR package have include prediction, for example, ROCR.hiv {ROCR}. My problem is how to produce prediction in SVM and to find

Dear Sir or Madam? I am a doctor of urology,and I am engaged in developing a nomogram of bladder cancer. May I ask for your help on below issue? I set up a dataset which include 317 cases. I got the Binary Logistic Regression model by SPSS.And then I try to reconstruct the model ?lrm(RECU~Complication+T.Num+T.Grade+Year+TS)? by R-Project,and try to internal validate the model through

Dear all, I am using the ROCR library to compute the AUC and also the Hmisc library to compute the C-index of a predictor and a group variable. The results of AUC and C-index are similar and give a value of about 0.57. The Wilcoxon p-value is <0.001! Why the AUC is showing small value and the p-value is high significant? The AUC is based on Wilcoxon calculation? Many thanks, Lina

Hi, there. Suppose I already have sensitivities and specificities. What is the quick R-function to calculate AUC for the ROC plot? There seem to be many R functions to calculate AUC. Thanks. Yulei [[alternative HTML version deleted]]

Dear all, I try to compare the performances of several parameters to diagnose lameness in dogs. I have several ROC curves from the same dataset. I plotted the ROC curves and calculated AUC with the ROCR package. I would like to compare the AUC. I used the following program I found on R-help archives : From: Bernardo Rangel Tura Date: Thu 16 Dec 2004 - 07:30:37 EST

Dear All, For logistic regression models: is it possible to use validate (rms package) to compute bias-corrected AUC, but have variable selection with AIC use step (or stepAIC, from MASS), instead of fastbw? More details: I've been using the validate function (in the rms package, by Frank Harrell) to obtain, among other things, bootstrap bias-corrected estimates of the AUC, when variable

Does anyone know how to get the C-index from a logistic model - not using the dataset that was used to train the model, but instead using a fresh dataset on the same model? I have a dataset of 400 points that I've split into two halves, one for training the logistic model, and the other for evaluating it. The structure is as follows: column headers are "got a loan" (dichotomous),

Guys, I used Random Forest with a couple of data sets I had to predict for binary response. In all the cases, the AUC of the training set is coming to be 1. Is this always the case with random forests? Can someone please clarify this? I have given a simple example, first using logistic regression and then using random forests to explain the problem. AUC of the random forest is coming out to be

Hello Is there an easy way, i.e. a function in a package, to calculate the area under the curve (AUC) for drug serum levels? Thanks for any advice -- Armin Goralczyk, M.D. -- Universit?tsmedizin G?ttingen Abteilung Allgemein- und Viszeralchirurgie Rudolf-Koch-Str. 40 39099 G?ttingen -- Dept. of General Surgery University of G?ttingen G?ttingen, Germany -- http://www.chirurgie-goettingen.de

Hi, I am trying to cross-validate a logistic regression model. I am using logistic regression model (lrm) of package Design. f <- lrm( cy ~ x1 + x2, x=TRUE, y=TRUE) val <- validate.lrm(f, method="cross", B=5) My class cy has values 0 and 1. "val" variable will give me indicators like slope and AUC. But, I also need the vector of predicted values of class variable

Hi, I was lately debugging parts of my 'colAUC' function in caTools package, and in a process looked into other packages for calculating Areas Under ROC Curves (AUC). To my surprise I found at least 6 other functions: * wilcox.test * AUC from ROC package, * performance from ROCR package, * auROC from limma package, * ROC from Epi package, * roc.area from verification