I use ROCR to plot multiple runs' performance. Using the sample code as example: # plot ROC curves for several cross-validation runs (dotted # in grey), overlaid by the vertical average curve and boxplots # showing the vertical spread around the average. data(ROCR.xval) pred <- prediction(ROCR.xval$predictions, ROCR.xval$labels) perf <- performance(pred,"tpr","fpr") plot(perf,col="grey82",lty=3) plot(perf,lwd=3,avg="vertical",spread.estimate="boxplot",add=TRUE) I can follow the code and plot without any problem. However, I don't know how to extract the averaged ROC area under curve value. Can someone help? Thanks. -- Waverley @ Palo Alto
waverley palo wrote:> > I use ROCR to plot multiple runs' performance. Using the sample code > as example: > > I can follow the code and plot without any problem. However, I don't > know how to extract the averaged ROC area under curve value. > >See the documentation of performance. Among many others, you can specifix 'auc' to be returned. Dieter -- View this message in context: http://www.nabble.com/ROCR-package-question-tp24655265p24656077.html Sent from the R help mailing list archive at Nabble.com.
Waverley, use @ (instead of $) to extract the slots from the performance object (it's S4 class system). HTH, Tobias On Sat, Jul 25, 2009 at 8:20 AM, Waverley<waverley.paloalto at gmail.com> wrote:> I use ROCR to plot multiple runs' performance. ?Using the sample code > as example: > > # plot ROC curves for several cross-validation runs (dotted > # in grey), overlaid by the vertical average curve and boxplots > # showing the vertical spread around the average. > data(ROCR.xval) > pred <- prediction(ROCR.xval$predictions, ROCR.xval$labels) > perf <- performance(pred,"tpr","fpr") > plot(perf,col="grey82",lty=3) > plot(perf,lwd=3,avg="vertical",spread.estimate="boxplot",add=TRUE) > > I can follow the code and plot without any problem. ?However, I don't > know how to extract the averaged ROC area under curve value. > > Can someone help? > > Thanks. > > -- > Waverley @ Palo Alto > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. >
Thanks for the reply. I am not sure I am following: 1. for the sample code. I tried perf at auc but get auc object not found 2. I am SPECIFICALLY interested in the averaged auc value of the multiple runs. How to get that out? I typed perf and it comes out as a list. 3. as for the plot using whisker plot to see the distribution of the multiple runs, the outliers outside the whisker is very annoying. How to get rid of the "outline" which is outside the whisker? I tried to use boxplot option and put in the following plot code as an option outline=FALSE and it did not work. Please help me with the specifics of the above 3 questions. Use code instead of description would be helpful. Thanks a lot in advance.>Waverley,>use @ (instead of $) to extract the slots from the performance object (it's S4 class system).>HTH, > TobiasOn Sat, Jul 25, 2009 at 8:20 AM, Waverley<waverley.paloalto at gmail.com> wrote:> I use ROCR to plot multiple runs' performance. Using the sample code > as example: > > # plot ROC curves for several cross-validation runs (dotted # in > grey), overlaid by the vertical average curve and boxplots # showing > the vertical spread around the average. > data(ROCR.xval) > pred <- prediction(ROCR.xval$predictions, ROCR.xval$labels) perf <- > performance(pred,"tpr","fpr") > plot(perf,col="grey82",lty=3) > plot(perf,lwd=3,avg="vertical",spread.estimate="boxplot",add=TRUE) > > I can follow the code and plot without any problem. However, I don't > know how to extract the averaged ROC area under curve value. > > Can someone help? > > Thanks. > > -- > Waverley @ Palo Alto > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. >-- Waverley @ Palo Alto
Thanks for the quick reply. That is very clear for my question 1, 2.
How about question 3? When I plot, is there way not to show the
whisker plot outliers for evaluating the multiple runs? I have tried
to put the option from boxplot command outline=FALSE, however, it did
not work.
Can you help?
Thanks again for your kind help.
Waverley, see help('performance-class') for a description of the slots.
Your AUCs will be in perf at y.values, which itself is a list (one list
element per run).
Thus, you can use functions like unlist or s/lapply to access them, e.g.
mean(unlist(perf at y.values))
Kind regards,
Tobias
On Sat, Jul 25, 2009 at 5:44 PM, Waverley<waverley.paloalto at gmail.com>
wrote:> Thanks for the reply. I am not sure I am following:
>
> 1. for the sample code. I tried perf at auc but get auc object not found
> 2. I am SPECIFICALLY interested in the averaged auc value of the
> multiple runs. How to get that out? I typed perf and it comes out as
> a list.
> 3. as for the plot using whisker plot to see the distribution of the
> multiple runs, the outliers outside the whisker is very annoying. How
> to get rid of the "outline" which is outside the whisker? I
tried to
> use boxplot option and put in the following plot code as an option
> outline=FALSE and it did not work.
>
> Please help me with the specifics of the above 3 questions. Use code
> instead of description would be helpful.
>
> Thanks a lot in advance.
>
>
>
>>Waverley,
>
>>use @ (instead of $) to extract the slots from the performance object
(it's S4 class system).
>
>>HTH,
>> Tobias
>
> On Sat, Jul 25, 2009 at 8:20 AM, Waverley<waverley.paloalto at
gmail.com> wrote:
>> I use ROCR to plot multiple runs' performance. Using the sample
code
>> as example:
>>
>> # plot ROC curves for several cross-validation runs (dotted # in
>> grey), overlaid by the vertical average curve and boxplots # showing
>> the vertical spread around the average.
>> data(ROCR.xval)
>> pred <- prediction(ROCR.xval$predictions, ROCR.xval$labels) perf
<-
>> performance(pred,"tpr","fpr")
>> plot(perf,col="grey82",lty=3)
>>
plot(perf,lwd=3,avg="vertical",spread.estimate="boxplot",add=TRUE)
>>
>> I can follow the code and plot without any problem. However, I
don't
>> know how to extract the averaged ROC area under curve value.
>>
>> Can someone help?
>>
>> Thanks.
>>
>> --
>> Waverley @ Palo Alto
>>
>> ______________________________________________
>> R-help at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
>> http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>
>
> --
> Waverley @ Palo Alto
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
--
Waverley @ Palo Alto