similar to: caret package: custom summary function in trainControl doesn't work with oob?

Gracias Carlos. No uso caret, pero lo miraré. Quoting Carlos Ortega <cof en qualityexcellence.es>: > Hola, > > Creo que si utilizas "caret" y en la función "trainControl()" defines "oob" > como criterio de randomización, puedes luego recuperar del objeto del > modelo, las predicciones individuales... > > Saludos, > Carlos Ortega >

I'm dealing with classification problems, and I'm trying to specify a custom scoring metric (recall at p, ROC, etc.) that depends on not just the class output but the probability estimates, so that caret::train can choose the optimal tuning parameters based on this metric. However, when I supply a trainControl summaryFunction, the data given to it contains only class predictions, so the

Hi list, I´m working on a predictive modeling task using the caret package. I found the best model parameters using the train() and trainControl() command. Now I want to evaluate my model and make predictions on a test dataset. I tried to follow the instructions in the manual and the vignettes but unfortunately I´m getting an error message I can`t figure out. Here is my code: rfControl <-

The following code fails to train a nnet model in a random dataset using caret: nR <- 700 nCol <- 2000 myCtrl <- trainControl(method="cv", number=3, preProcOptions=NULL, classProbs = TRUE, summaryFunction = twoClassSummary) trX <- data.frame(replicate(nR, rnorm(nCol))) trY <- runif(1)*trX[,1]*trX[,2]^2+runif(1)*trX[,3]/trX[,4] trY <-

Dear all, I am using the "caret" Package for predictors selection with a randomForest model. The following is the train function: rfFit<- train(x=trainRatios, y=trainClass, method="rf", importance = TRUE, do.trace = 100, keep.inbag = TRUE, tuneGrid = grid, trControl=bootControl, scale = TRUE, metric = "ROC") I wanted to use ROC as the metric for variable

I am used to packages like e1071 where you have a tune step and then pass your tunings to train. It seems with caret, tuning and training are both handled by train. I am using train and trainControl to find my hyper parameters like so: MyTrainControl=trainControl( method = "cv", number=5, returnResamp = "all", classProbs = TRUE ) rbfSVM <- train(label~., data =

Hi all, I run R 2.11.1 under ubuntu 10.10 and caret version 2.88. I use the caret package to compare different models on a dataset. In order to compare their different performances I would like to use the same data partitions for every models. I understand that using a LGOCV or a boot type re-sampling method along with the "index" argument of the trainControl function, one is able to

I'm using caret to assess classifier performance (and it's great!). However, I've found that my results differ between R2.* and R3.* - reported accuracies are reduced dramatically. I suspect that a code change to kernlab ksvm may be responsible (see version 5.16-24 here: http://cran.r-project.org/web/packages/caret/news.html). I get very different results between caret_5.15-61 +

Muy buenas, ¿sabe alguien cómo obtener las predicciones sobre el out of bag que hace randomForest? Manuel . -- Dr Manuel Mendoza Department of Biogeography and Global Change National Museum of Natural History (MNCN) Spanish Scientific Council (CSIC) C/ Serrano 115bis, 28006 MADRID Spain

I have carefully read the CARET documentation at: http://caret.r-forge.r-project.org/training.html, the vignettes, and everything is quite clear (the examples on the website help a lot!), but I am still a confused about the relationship between two arguments to trainControl: "method" "index" and the interplay between trainControl and the data splitting functions in caret

Hello, We have a system which creates thousands of regression/classification models and in cases where we have only one input variable NaiveBayes throws an error. Maybe I am mistaken and I shouldn't expect to have a model with only one input variable. We use R version 2.6.0 (2007-10-03). We use caret (v4.1.19), but have tested similar code with klaR (v.0.5.8), because caret relies on

Hello all, I would like to use the train function from the caret package to train a svm with a spectral kernel from the kernlab package. Sadly a svm with spectral kernel is not among the many methods in caret... using caret to train svmRadial: ------------------ library(caret) library(kernlab) data(iris) TrainData<- iris[,1:4] TrainClasses<- iris[,5] set.seed(2)

The documentation for caret::train shows a list of parameters that one can tune for each method classification/regression method. For example, for the method randomForest one can tune mtry in the call to train. But the function call to train random forests in the original package has many other parameters, e.g. sampsize, maxnodes, etc. Is there **any** way to access these parameters using train

Hy, I got the following problem when trying to build a rpart model and using everything but LOOCV. Originally, I wanted to used k-fold partitioning, but every partitioning except LOOCV throws the following warning: ---- Warning message: In nominalTrainWorkflow(dat = trainData, info = trainInfo, method = method, : There were missing values in resampled performance measures. ----- Below are some

I want to use the caret package and found out about the timingSamps obtion to obtain the time which is needed to predict results. But, as soon as I set a value for this option, the whole model generation fails. Check this example: ------------------------- library(caret) tc=trainControl(method='LGOCV', timingSamps=10) tcWithout=trainControl(method='LGOCV')

Greetings, I have been exploring the use of the caret package to conduct some plsda modeling. Previously, I have come across methods that result in a R2 and Q2 for the model. Using the 'iris' data set, I wanted to see if I could accomplish this with the caret package. I use the following code: library(caret) data(iris) #needed to convert to numeric in order to do regression #I

Dear all, I have used following code but everytime I encounter a problem of not having coefficients for all the variables in the predictor set. # code rm(list=ls()) library(caret) # generating response and design matrix X<-matrix(rnorm(50*100),nrow=50) y<-rnorm(50*1) # Applying caret package con<-trainControl(method="cv",number=10) data<-NULL data<- train(X,y,

Usually when using raw glmnet I let the implementation choose the lambdas. However when training via caret::train the lambda values are predetermined. Is there any way to have caret defer the lambda choices to caret::train and thus choose the optimal lambda dynamically? -- Yang Zhang http://yz.mit.edu/

I want to use caret to build a model with an algorithm that actually has no parameters to find. How do I stop it from repeatedly building the same model 25 times? library(caret) data(mdrr) LOGISTIC_model <- train(mdrrDescr,mdrrClass ,method='glm' ,family=binomial(link="logit") ) LOGISTIC_model 528

Hello, I have some time series applications, where i have a large set of X variables (hundreds) and thousands of time data points (sampling every minute). I like to use the caret package to support the analysis, variable selection and model selection. However, reading the documentation, it looks like caret uses resampling methods. Not sure if these methods work with time series, as you