R help - May 2009 - ROCR: auc and logarithm plot

Hi,
I am quite new to R and I have two questions regarding ROCR.

1. I have tried to understand how to extract area-under-curve value by looking
at the ROCR document and googling. Still I am not sure if I am doing the right
thing. Here is my code, is "auc1" the auc value?
"
pred1 <- prediction(resp1,label1)

perf1 <- performance(pred1,"tpr","fpr")
plot( perf1, type="l",col=1 )

auc1 <- performance(pred1,"auc")
auc1 <- auc1@y.values[[2]]
"

2. I have to compare two models that have very close ROCs. I'd like to have
a more distinguishable plot of the ROCs. So is it possible to have a logarithm
FP axis which might probably separate them well? Or zoom in the part close to
the leftup corner of ROC plot? Or any other ways to make the ROCs more separate?

Thanks and regards!

--Tim



      
	[[alternative HTML version deleted]]

> 1. I have tried to understand how to extract area-under-curve value by
looking at the ROCR document and googling. Still I am not sure if I am doing the
right thing. Here is my code, is "auc1" the auc value?
> "
> pred1 <- prediction(resp1,label1)
>
> perf1 <- performance(pred1,"tpr","fpr")
> plot( perf1, type="l",col=1 )
>
> auc1 <- performance(pred1,"auc")
> auc1 <- auc1 at y.values[[2]]
> "

If you have only one set of predictions and matching class labels, it
would be in auc1 at y.values[[1]].
If you have multiple sets (as from cross-validation or bootstrapping),
the AUCs would be in auc1 at y.values[[1]], auc1 at y.values[[2]], etc.
You can collect all of them for example by unlist(perf at y.values).

Btw, you can use str(auc1) to see the structure of objects.

> 2. I have to compare two models that have very close ROCs. I'd like to
have a more distinguishable plot of the ROCs. So is it possible to have a
logarithm FP axis which might probably separate them well? Or zoom in the part
close to the leftup corner of ROC plot? Or any other ways to make the ROCs more
separate?

To "zoom in" to a specific part:
plot(perf1, xlim=c(0,0.2), ylim=c(0.7,1))
plot(perf2, add=TRUE, lty=2, col='red')

If you want logarithmic axes (though I wouldn't personally do this for
a ROC plot), you can set up an empty canvas and add ROC curves to it:
plot(1,1, log='x', xlim=c(0.001,1), ylim=c(0,1), type='n')
plot(perf, add=TRUE)

You can adjust all components of the performance plots. See
?plot.performance and the examples in this slide deck:
http://rocr.bioinf.mpi-sb.mpg.de/ROCR_Talk_Tobias_Sing.ppt

Hope that helps,
  Tobias