search for: savepredictions

Displaying 4 results from an estimated 4 matches for "savepredictions".

2017 Oct 16
1
ROC curve for each fold in one plot
Hi all, I have tried a 5 fold cross validation using caret package with random forest method on iris dataset as example. Then I need ROC curve for each fold: > set.seed(1) > train_control <- trainControl(method="cv", number=5,savePredictions = TRUE,classProbs = TRUE) > output <- train(Species~., data=iris, trControl=train_control, method="rf") > library(pROC) > selectedIndices <- output$pred$Resample == "Fold1" > plot.roc(output$pred$obs[selectedIndices],output$pred$setosa[selectedIndice...
2017 Dec 02
0
How can you find the optimal number of values to randomly sample to optimize random forest classification without trial and error?
...ation algorithm. My goal of this post is to figure out how to identify the optimal "s" value so that the highest ROC is achieved. I am using "rf" from the caret package to do classification. library(caret) fitControl <-trainControl(method = "LOOCV", classProbs = T, savePredictions = T) model <- train(status ~ ., data = dataset, method = "rf", trControl = fitControl) How can I automate it to start "s" at 5000, change it to another value, and based on the change in ROC, keep changing "s" to work towards the best possible "s" value?...
2017 Oct 22
0
Test set and Train set in Caret package train function
Hey all, Does anyone know how we can get train set and test set for each fold of 5 fold cross validation in Caret package? Imagine if I want to do cross validation by random forest method, I do the following in Caret: set.seed(12) train_control <- trainControl(method="cv", number=5,savePredictions = TRUE) rfmodel <- train(Species~., data=iris, trControl=train_control, method="rf") first_holdout <- subset(rfmodel$pred, Resample == "Fold1") str(first_holdout) 'data.frame': 90 obs. of 5 variables: $ pred : Factor w/ 3 levels "setosa","versico...
2013 Nov 15
1
Inconsistent results between caret+kernlab versions
I'm using caret to assess classifier performance (and it's great!). However, I've found that my results differ between R2.* and R3.* - reported accuracies are reduced dramatically. I suspect that a code change to kernlab ksvm may be responsible (see version 5.16-24 here: http://cran.r-project.org/web/packages/caret/news.html). I get very different results between caret_5.15-61 +