similar to: ROCR package finding maximum accuracy and optimal cutoff point

All, I'm trying again with a slightly more generic version of my first question. I can extract the plotted values from hist(), boxplot(), and even plot.randomForest(). Observe: # get some data dat <- rnorm(100) # grab histogram data hdat <- hist(dat) hdat #provides details of the hist output #grab boxplot data bdat <- boxplot(dat) bdat #provides details of the boxplot

Hi, I'm working with support vector machine for the classification purpose, and I have a problem about the accuracy of prediction. I divided my data set in train (1/3 of enteire data set) and test (2/3 of data set) using the "sample" function. Each time I perform the svm model I obtain different result, according with the result of the "sample" function. I would like

I am able to split my df into two like so: dataset <- trainset index <- 1:nrow(dataset) testindex <- sample(index, trunc(length(index)*30/100)) trainset <- dataset[-testindex,] testset <- dataset[testindex,-1] So I have the index information, how could I re-combine the data using that back into a single df? I tried what I thought might work, but failed with:

Dear List, I have encountered an odd problem that I cannot understand. It stems from the calculation of true and false positives based on two input vectors x and y based on different thresholds of x, extracted using the quantile function. I am in certain cases getting different values of true positives for the same threshold value when the threshold was found under different quantiles (e.g. the

Dear list I'm testing a predictor and I produced nice performance plots with ROCR package utilizing the 3 standard command pred <- prediction(predictions, labels) perf <- performance(pred, measure = "tpr", x.measure = "fpr") plot(perf, col=rainbow(10)) The pred object and the perfo object are S4 with the following slots An object of class "performance"

I am trying to use package ROCR to analyze classification accuracy, unfortunately there are some problems right at the beginning. Question 1) When I try to run demo I am getting the following error message > library(ROCR) > demo(ROCR) > if(dev.cur() <= 1) .... [TRUNCATED] Error in get(getOption("device")) : wrong first argument When I issue the command > dev.cur() it

Hi, My questions concerns the ROCR package and I hope somebody here on the list can help - or point me to some better place. When evaluating a model's performane, like this: pred1 <- predict(model, ..., type="response") pred2 <- prediction(pred1, binary_classifier_vector) perf <- performance(pred, "sens", "spec") (Where "prediction" and

I am new to R, so I am sure I am making a simple mistake. I am including complete information in hopes someone can help me. Basically my data in R looks good, I write it to a file, and every value is off by 1. Here is my flow: > str(prediction) Factor w/ 10 levels "0","1","2","3",..: 3 1 10 10 4 8 1 4 1 4 ... - attr(*, "names")= chr

I use ROCR to plot multiple runs' performance. Using the sample code as example: # plot ROC curves for several cross-validation runs (dotted # in grey), overlaid by the vertical average curve and boxplots # showing the vertical spread around the average. data(ROCR.xval) pred <- prediction(ROCR.xval$predictions, ROCR.xval$labels) perf <- performance(pred,"tpr","fpr")

Hello All,  I have used logistic regression glm in R and I am evaluating two models both learned with glm but with different predictors. model1 <- glm (Y ~ x4+ x5+ x6+ x7, data = dat, family = binomial(link=logit))model2 <- glm (Y~ x1 + x2 +x3 , data = dat, family = binomial(link=logit))  and I would like to compare these two models based on the prediction that I get from each model: pred1 =

Dear R-philes, I am plotting ROC curves for several cross-validation runs of a classifier (using the function below). In addition to the average AUC, I am interested in obtaining a confidence interval for the average AUC. Is there a straightforward way to do this via the ROCR package? plot_roc_curve <- function(roc.dat, plt.title) { #print(str(vowel.ROC)) pred <-

Hi, I am quite new to R and I have two questions regarding ROCR. 1. I have tried to understand how to extract area-under-curve value by looking at the ROCR document and googling. Still I am not sure if I am doing the right thing. Here is my code, is "auc1" the auc value? " pred1 <- prediction(resp1,label1) perf1 <- performance(pred1,"tpr","fpr") plot(

Hi, I'm involved in a bioinformatics project at my university, and we're doing a comparison paper between some methods of classification of nc-RNA. I've been encharged of ploting the ROC curves' graphs. But I'm new on working with R and I'm having some difficulty with the prediction-class. I don't get where the values of ROCR.simple$predictions, for example, came from

Dear list   I have 5 markers that can be used to detect an infection in combination. Could you please advise me how to use functions in ROCR/ other package to produce the ROC curve for a combination of markers?   I have used the following to get ROC statistics for each marker. pred <- prediction(y$marker1, y$infectn) perf <-performance(pred,"tpr","fpr")

> x <- train[,c( 2:18, 20:21, 24, 27:31)] > y <- train$out > > svm.pr <- svm(x, y, probability = TRUE, method="C-classification", kernel="radial", cost=bestc, gamma=bestg, cross=10) > > pred <- predict(svm.pr, valid[,c( 2:18, 20:21, 24, 27:31)], decision.values = TRUE, probability = TRUE) > attr(pred, "decision.values")[1:4,]

Dear List,   I am new to ROC analysis and the package ROCR. I want to compute the confidence intervals of sensitivity and specificity for a given cutoff value. I have used the following to calculate sensitivity and specificity:   data(ROCR.simple) pred <- prediction(ROCR.simple$predictions, ROCR.simple$labels)   se.sp <- function (cutoff, performance) {     sens <-

I'm trying to use ROCR to create a simple cutoff vs recall plot (recall at p) on the example ROCR.simple dataset: library(ROCR) data(ROCR.simple) pred <- prediction(ROCR.simple$predictions, ROCR.simple$labels) perf <- performance(pred, "rec") plot(perf) But R crashes on me on the last line. I'm using R 2.14.1, ROCR 1.0-4. ?Any ideas? Thanks in advance. -- Yang Zhang

Hi everybody, I have a data set with a value and a status (positive or negative case) and I want make a ROC Analysis. So, with ROCR Package, I have got the ROC curve (True Positive Fraction [tpf] according 1-True Negative Fraction [1-tnf]). http://r.789695.n4.nabble.com/file/n4469203/01.png But, now I want a new graphic which show the sum of true positive fraction and true negative fraction

I have never been able to get class probabilities to work and I am relatively new to using these tools, and I am looking for some insight as to what may be wrong. I am using caret with kernlab/ksvm. I will simplify my problem to a basic data set which produces the same problem. I have read the caret vignettes as well as documentation for ?train. I appreciate any direction you can give. I

Hi everybody, I am having a problem building a ROC curve with my data using the ROCR package. I have 10 lists of proteins such as attached (proteinlist.xls). each of the lists was calculated with a different p-value. The goal is to find the optimal p-value for the highest number of true positives as well as lowaest number of false positives. As far as I understood the explanations from the