search for: loocv

Hello, I have the following function that receives a "function pointer" formal parameter name "fnc": loocv <- function(data, fnc) { n <- length(data.x) score <- 0 for (i in 1:n) { x_i <- data.x[-i] y_i <- data.y[-i] yhat <- fnc(x=x_i,y=y_i) score <- score + (y_i - yhat)^2 } score <- score/n return(score) } I would like to...

Hy, I got the following problem when trying to build a rpart model and using everything but LOOCV. Originally, I wanted to used k-fold partitioning, but every partitioning except LOOCV throws the following warning: ---- Warning message: In nominalTrainWorkflow(dat = trainData, info = trainInfo, method = method, : There were missing values in resampled performance measures. ----- Below are som...

Hello, How can I do a cross validation in R? Thank You!

I am trying to write a wrapper for the last example in help(cv.glm) that deals with leave-one-out-cross-validation (LOOCV) for a logistic model. This wrapper will be used as part of a bigger program. Here is my wrapper funtion : logistic.LOOCV.err <- function( formu=NULL, data=NULL ){ cost.fn <- function(cl, pred) mean( abs(cl-pred) > 0.5 ) glmfit <- glm( formula=formu, data=data, family=...

Good afternoon. I would like to know if there is any function in R to do LOOCV with these classifiers: 1)SVM 2)Neural Networks 3)C4.5 ( J48) 4)KNN Thanks a lot! [[alternative HTML version deleted]]

Can someone help me with performing leave-out-one cross validation using R (model built is a Cox model)? Thanks. --------------------------------------------- David Verbel, MPH Senior Biostatistician Aureon Biosciences 28 Wells Avenue Yonkers, NY 10701 Phone: (914) 377-4021 Fax: (914) 377-4001 --------------------------------------------- [[alternative HTML version deleted]]

Hello all. I have this code: myLOOCV <- function(myformula, data) { Y <- all.vars(myformula)[1] Scores<- numeric(length(data[,1])) for (i in 1:length(data[,1])) { train <- data[-i,] test <- data[i,] myModel <- lrm(myformula, train) Scores[i] <- predict(myModel, test,type...

Dear all, not really a R question but: If I want to check for the classification accuracy of a LDA with previous PCA for dimensionality reduction by means of the LOOCV method: Is it ok to do the PCA on the WHOLE dataset ONCE and then run the LDA with the CV option set to TRUE (runs LOOCV) -- OR-- do I need - to compute for each 'test-bag' (the n-1 observations) a PCA (my.princomp.1), - then run the LDA on the test-bag scores (-> my.lda.1) - then c...

..."Classification Using Generalized Partial Least Squares", Beiying Ding, Robert Gentleman, Bioconductor Project Working Papers, year 2004, paper 5 Section 2.6 Assessing Prediction: Cite: "The optimal number of PLS components is selected by choosing that value of K which minimizes LOOCV error rate for the training set." and in section 3.1.3 Colon data, subsection: Random splitting Cite: "Due to the instability of LOOCV error rates for data with few samples and many covariates, comparison of various classifiers based solely on LOOCV classification errors may not be r...

...x) return(prediction) } x <- matrix(rnorm(5100),102,50) rownames(x) <- paste('a',1:102,sep='') colnames(x) <- paste('b',1:50,sep='') y <- factor(sample(1:2,102,replace=T)) results <- crossval(x,y,theta.fit,theta.predict) # LOOCV ----------------------------------------------------------------------------------------------------------- I get the following error: Error in scale(newdata[, object$scaled, drop = FALSE], center = object$x.scale$"scaled:center", : (subscript) logical subscript too long It seems...

Hello all, Is there any function in R by which I can calculate PRESS and P2 statistics for linear regression in R? Thanks Alex [[alternative HTML version deleted]]

...e being used in the classification algorithm. My goal of this post is to figure out how to identify the optimal "s" value so that the highest ROC is achieved. I am using "rf" from the caret package to do classification. library(caret) fitControl <-trainControl(method = "LOOCV", classProbs = T, savePredictions = T) model <- train(status ~ ., data = dataset, method = "rf", trControl = fitControl) How can I automate it to start "s" at 5000, change it to another value, and based on the change in ROC, keep changing "s" to work towards t...