Displaying 14 results from an estimated 14 matches for "classprobs".
2013 Nov 15
1
Inconsistent results between caret+kernlab versions
...rom https
df <- source_data('https://dl.dropboxusercontent.com/u/47973221/data.csv', sep=',')
require(caret)
svm.m1 <- train(df[,-1],df[,1],method='svmRadial',metric='Kappa',tunelength=5,trControl=trainControl(method='repeatedcv', number=10, repeats=10, classProbs=TRUE))
svm.m1
sessionInfo()
### Results - R2.15.2
> svm.m1
1241 samples
7 predictors
10 classes: ?O27479?, ?O31403?, ?O32057?, ?O32059?, ?O32060?, ?O32078?, ?O32089?, ?O32663?, ?O32668?, ?O32676?
No pre-processing
Resampling: Cross-Validation (10 fold, repeated 10 times)
Summary of sa...
2011 Aug 28
1
Trying to extract probabilities in CARET (caret) package with a glmStepAIC model
...encountered a problem:
I am working with a dataset that include 4 predictor variables in Descr and a two-category outcome in Categ (codified as a factor).
Everything was working fine I got the results, confussion matrix etc.
BUT for obtaining the AUC and predicted probabilities I had to add " classProbs = TRUE," in the trainControl. Thereafter everytime I run train I get this message:
"undefined columns selected"
I copy the syntax:
fitControl <- trainControl(method = "cv", number = 10, classProbs = TRUE,returnResamp = "all", verboseIter = FALSE)
glmFit <-...
2011 May 12
2
Can ROC be used as a metric for optimal model selection for randomForest?
...y=trainClass, method="rf", importance = TRUE, do.trace = 100, keep.inbag = TRUE,
tuneGrid = grid, trControl=bootControl, scale = TRUE, metric = "ROC")
I wanted to use ROC as the metric for variable selection. I know that this works with the logit model by making sure that classProbs = TRUE and summaryFunction = twoClassSummary in the trainControl function. However if I do the same with randomForest, I get a warning saying that
"In train.default(x = trainPred, y = trainDep, method = "rf", :
The metric "ROC" was not in the result set. Accuracy will...
2012 Nov 23
1
caret train and trainControl
...ave a tune step and then pass your tunings to train.
It seems with caret, tuning and training are both handled by train.
I am using train and trainControl to find my hyper parameters like so:
MyTrainControl=trainControl(
method = "cv",
number=5,
returnResamp = "all",
classProbs = TRUE
)
rbfSVM <- train(label~., data = trainset,
method="svmRadial",
tuneGrid = expand.grid(.sigma=c(0.0118),.C=c(8,16,32,64,128)),
trControl=MyTrainControl,
fit = FALSE
)
Once this returns my ideal para...
2023 May 09
1
RandomForest tuning the parameters
...uot;ntree"),
> class = rep("numeric", 2), label = c("maxnodes", "ntree"))
> >
> > customRF$grid <- function(x, y, len = NULL, search = "grid") {}
> >
> > customRF$fit <- function(x, y, wts, param, lev, last, weights,
> classProbs, ...) {
> >
> > randomForest(x, y, maxnodes = param$maxnodes, ntree=param$ntree, ...)
> >
> > }
> >
> > customRF$predict <- function(modelFit, newdata, preProc = NULL,
> submodels = NULL)
> >
> > predict(modelFit, newdata)
> >
> > c...
2012 Nov 29
1
Help with this error "kernlab class probability calculations failed; returning NAs"
...set)
testindex <- sample(index, trunc(length(index)*30/100))
trainset <- dataset[-testindex,]
testset <- dataset[testindex,-1]
## TUNE caret / kernlab
set.seed(1)
MyTrainControl=trainControl(
method = "repeatedcv",
number=10,
repeats=5,
returnResamp = "all",
classProbs = TRUE
)
## MODEL
rbfSVM <- train(outcome~., data = trainset,
method="svmRadial",
preProc = c("scale"),
tuneLength = 10,
trControl=MyTrainControl,
fit = FALSE
)
There were 50 or mor...
2013 Mar 06
1
CARET and NNET fail to train a model when the input is high dimensional
The following code fails to train a nnet model in a random dataset using
caret:
nR <- 700
nCol <- 2000
myCtrl <- trainControl(method="cv", number=3, preProcOptions=NULL,
classProbs = TRUE, summaryFunction = twoClassSummary)
trX <- data.frame(replicate(nR, rnorm(nCol)))
trY <- runif(1)*trX[,1]*trX[,2]^2+runif(1)*trX[,3]/trX[,4]
trY <- as.factor(ifelse(sign(trY)>0,'X1','X0'))
my.grid <- createGrid(method.name, grid.len, data=trX)
my.mode...
2017 Oct 16
1
ROC curve for each fold in one plot
Hi all,
I have tried a 5 fold cross validation using caret package with random forest method on iris dataset as example. Then I need ROC curve for each fold:
> set.seed(1)
> train_control <- trainControl(method="cv", number=5,savePredictions = TRUE,classProbs = TRUE)
> output <- train(Species~., data=iris, trControl=train_control, method="rf")
> library(pROC)
> selectedIndices <- output$pred$Resample == "Fold1"
> plot.roc(output$pred$obs[selectedIndices],output$pred$setosa[selectedIndices]) > selected...
2023 May 08
1
RandomForest tuning the parameters
...ta.frame(parameter = c("maxnodes", "ntree"), class = rep("numeric", 2), label = c("maxnodes", "ntree"))?
customRF$grid <- function(x, y, len = NULL, search = "grid") {}
customRF$fit <- function(x, y, wts, param, lev, last, weights, classProbs, ...) {
?randomForest(x, y, maxnodes = param$maxnodes, ntree=param$ntree, ...)
}
customRF$predict <- function(modelFit, newdata, preProc = NULL, submodels = NULL)
predict(modelFit, newdata)
customRF$prob <- function(modelFit, newdata, preProc = NULL, submodels = NULL)
? predict(modelFit...
2011 Dec 22
0
randomforest and AUC using 10 fold CV - Plotting results
...s[[1]]
legend("bottomright",legend=c(paste("Random Forests
(AUC=",formatC(auc1,digits=4,format="f"),")",sep="")),
col=c("red"), lty=1)
#Cross validation using 10 fold CV:
ctrl <- trainControl(method = "cv", classProbs = TRUE, summaryFunction =
twoClassSummary)
set.seed(1)
rfEstimate <- train(factor(Species) ~ .,data = iris, method = "rf",
metric = "ROC", tuneGrid = data.frame(.mtry = 2), trControl = ctrl)
rfEstimate
How can i plot the results from the cross validation on the previous...
2012 Feb 10
1
Custom caret metric based on prob-predictions/rankings
I'm dealing with classification problems, and I'm trying to specify a
custom scoring metric (recall at p, ROC, etc.) that depends on not just
the class output but the probability estimates, so that caret::train
can choose the optimal tuning parameters based on this metric.
However, when I supply a trainControl summaryFunction, the data given
to it contains only class predictions, so the
2013 Feb 10
1
Training with very few positives
...ning a classifier
with very few positives?
I currently have the following setup:
========================================
library(caret)
tmp <- createDataPartition(Y, p = 9/10, times = 3, list = TRUE)
myCtrl <- trainControl(method = "boot", index = tmp, timingSamps = 2,
classProbs = TRUE, summaryFunction = twoClassSummary)
RFmodel <- train(X,Y,method='rf',trControl=myCtrl,tuneLength=1,
metric="ROC")
SVMmodel <- train(X,Y,method='svmRadial',trControl=myCtrl,tuneLength=3,
metric="ROC")
KNNmodel <- train(X,Y,method='...
2017 Dec 02
0
How can you find the optimal number of values to randomly sample to optimize random forest classification without trial and error?
...in the classification algorithm. My
goal of this post is to figure out how to identify the optimal "s" value so
that the highest ROC is achieved. I am using "rf" from the caret package to
do classification.
library(caret)
fitControl <-trainControl(method = "LOOCV", classProbs = T, savePredictions = T)
model <- train(status ~ ., data = dataset, method = "rf", trControl =
fitControl)
How can I automate it to start "s" at 5000, change it to another value, and
based on the change in ROC, keep changing "s" to work towards the best
possible &...
2010 Oct 22
2
Random Forest AUC
Guys,
I used Random Forest with a couple of data sets I had to predict for binary
response. In all the cases, the AUC of the training set is coming to be 1.
Is this always the case with random forests? Can someone please clarify
this?
I have given a simple example, first using logistic regression and then
using random forests to explain the problem. AUC of the random forest is
coming out to be