David martin
2011-Dec-22 10:26 UTC
[R] randomforest and AUC using 10 fold CV - Plotting results
Here is a snippet to show what i'm trying to do. library(randomForest) library(ROCR) library(caret) data(iris) iris <- iris[(iris$Species != "setosa"),] fit <- randomForest(factor(Species) ~ ., data=iris, ntree=50) train.predict <- predict(fit,iris,type="prob")[,2] plot(performance(prediction(train.predict,factor(iris$Species)),"tpr","fpr"),col = "red") #As expected AUC is 1 because we are using the same dataset to validate auc1 <- performance(prediction(train.predict,factor(iris$Species)),"auc")@y.values[[1]] legend("bottomright",legend=c(paste("Random Forests (AUC=",formatC(auc1,digits=4,format="f"),")",sep="")), col=c("red"), lty=1) #Cross validation using 10 fold CV: ctrl <- trainControl(method = "cv", classProbs = TRUE, summaryFunction = twoClassSummary) set.seed(1) rfEstimate <- train(factor(Species) ~ .,data = iris, method = "rf", metric = "ROC", tuneGrid = data.frame(.mtry = 2), trControl = ctrl) rfEstimate How can i plot the results from the cross validation on the previous ROC plot ? thanks, david