R help - Feb 2011 - ROC from R-SVM?

*Hi,

*Does anyone know how can I show an *ROC curve for R-SVM*? I understand in
R-SVM we are not optimizing over SVM cost parameter. Any example ROC for
R-SVM code or guidance can be really useful.

Thanks, Angel.

	[[alternative HTML version deleted]]

The objects functions for kernel methods are unrelated to the area
under the ROC curve. However, you can try to choose the cost and
kernel parameters to maximize the ROC AUC.

See the caret package, specifically the train function.

Max

On Mon, Feb 21, 2011 at 5:34 PM, Angel Russo <angerusso1980 at gmail.com>
wrote:
> *Hi,
>
> *Does anyone know how can I show an *ROC curve for R-SVM*? I understand in
> R-SVM we are not optimizing over SVM cost parameter. Any example ROC for
> R-SVM code or guidance can be really useful.
>
> Thanks, Angel.
>
> ? ? ? ?[[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>


-- 

Max

In addition's to Max's suggestion about caret, look at ROCR which
visualizes ROC charts for any binary classifier.  I have an example of
e1071::SVN and ROCR here

https://heuristically.wordpress.com/2009/12/23/compare-performance-machine-learning-classifiers-r/



-----Original Message-----
From: r-help-bounces at r-project.org [mailto:r-help-bounces at r-project.org]
On Behalf Of Angel Russo
Sent: Monday, February 21, 2011 3:34 PM
To: r-help at r-project.org
Subject: [R] ROC from R-SVM?

*Hi,

*Does anyone know how can I show an *ROC curve for R-SVM*? I understand in
R-SVM we are not optimizing over SVM cost parameter. Any example ROC for
R-SVM code or guidance can be really useful.

Thanks, Angel.

Hi Max and Andrew,

Thanks so much for your reply. Indeed I found your link last night using
steps shown below.

My first question is if the following two steps are right and AUC is 51% as
shown below.

My seond question is that currently I am using cost parameter=1 (the default
in R-SVM; http://www.stanford.edu/group/wonglab/RSVMpage/R-SVM.html). To
improve the AUC from ROC curve, can I can optimize the SVM cost function
(instead of keeping it fixed a 1) to get the mimimum LOO error in training
cross.validation and then draw the ROC from decision.values as Step2 below
using a cost parameter that gave mimimum cross.validation error in the
training data. Is that right?

Many thanks.
----------

*Step 1: For obtaining ROC curve of test data I turned on "prob=T"
option:*
> svmres.prob <- svm(traindx[,resrsvm$SelInd], as.factor(traindy),
decision.values = TRUE)
> svmpred.prob <- predict(svmres.prob, testdx[,resrsvm$SelInd],
decision.values = TRUE)
> print(confusionMatrix(svmpred.prob,testdy))
Confusion Matrix and Statistics

           Reference
Prediction  Resistant Sensitive
  Resistant         5        13
  Sensitive        37        88

             Accuracy : 0.6503


*Step 2: Actual ROC plot command using output from above and plot attached
as well as pdf (I am assuming the following says the AUC is
51.4):*
> library(ROCR)
> svm.roc <- prediction(attributes(svmpred.prob)$decision.values, testdy)
> svm.auc <- performance(svm.roc, 'tpr', 'fpr')
> aucsvm <- performance(svm.roc, 'auc')
> pdf(file="roc_curve_rsvm_decval.pdf")
> plot(svm.auc)
> print(str(aucsvm))
> print(str(aucsvm))
Formal class 'performance' [package "ROCR"] with 6 slots
  ..@ x.name      : chr "None"
  ..@ y.name      : chr "Area under the ROC curve"
  ..@ alpha.name  : chr "none"
  ..@ x.values    : list()
  ..@ y.values    :List of 1
  .. ..$ : num 0.514
  ..@ alpha.values: list()

---------------------------



On Tue, Feb 22, 2011 at 4:23 PM, Andrew Ziem <AZiem@us.ci.org> wrote:
> In addition's to Max's suggestion about caret, look at ROCR which
> visualizes ROC charts for any binary classifier.  I have an example of
> e1071::SVN and ROCR here
>
>
>
https://heuristically.wordpress.com/2009/12/23/compare-performance-machine-learning-classifiers-r/
>
>
>
> -----Original Message-----
> From: r-help-bounces@r-project.org [mailto:r-help-bounces@r-project.org]
> On Behalf Of Angel Russo
> Sent: Monday, February 21, 2011 3:34 PM
> To: r-help@r-project.org
> Subject: [R] ROC from R-SVM?
>
> *Hi,
>
> *Does anyone know how can I show an *ROC curve for R-SVM*? I understand in
> R-SVM we are not optimizing over SVM cost parameter. Any example ROC for
> R-SVM code or guidance can be really useful.
>
> Thanks, Angel.
>
>
>
	[[alternative HTML version deleted]]