search for: savepredict

Hi all, I have tried a 5 fold cross validation using caret package with random forest method on iris dataset as example. Then I need ROC curve for each fold: > set.seed(1) > train_control <- trainControl(method="cv", number=5,savePredictions = TRUE,classProbs = TRUE) > output <- train(Species~., data=iris, trControl=train_control, method="rf") > library(pROC) > selectedIndices <- output$pred$Resample == "Fold1" > plot.roc(output$pred$obs[selectedIndices],output$pred$setosa[selectedIn...

...ation algorithm. My goal of this post is to figure out how to identify the optimal "s" value so that the highest ROC is achieved. I am using "rf" from the caret package to do classification. library(caret) fitControl <-trainControl(method = "LOOCV", classProbs = T, savePredictions = T) model <- train(status ~ ., data = dataset, method = "rf", trControl = fitControl) How can I automate it to start "s" at 5000, change it to another value, and based on the change in ROC, keep changing "s" to work towards the best possible "s" valu...

Hey all, Does anyone know how we can get train set and test set for each fold of 5 fold cross validation in Caret package? Imagine if I want to do cross validation by random forest method, I do the following in Caret: set.seed(12) train_control <- trainControl(method="cv", number=5,savePredictions = TRUE) rfmodel <- train(Species~., data=iris, trControl=train_control, method="rf") first_holdout <- subset(rfmodel$pred, Resample == "Fold1") str(first_holdout) 'data.frame': 90 obs. of 5 variables: $ pred : Factor w/ 3 levels "setosa","ver...

I'm using caret to assess classifier performance (and it's great!). However, I've found that my results differ between R2.* and R3.* - reported accuracies are reduced dramatically. I suspect that a code change to kernlab ksvm may be responsible (see version 5.16-24 here: http://cran.r-project.org/web/packages/caret/news.html). I get very different results between caret_5.15-61 +