Wade Wall
2006-Nov-16 15:04 UTC
[R] X-fold cross validation function for discriminant analysis
Hi all, I ran a discriminant analysis with some data and want to get a general idea of prediction error rate. Some have suggested using X-fold cross validation procedure. Anyone know if there is a function for this in R? Thanks, Wade [[alternative HTML version deleted]]
Wensui Liu
2006-Nov-16 16:37 UTC
[R] X-fold cross validation function for discriminant analysis
how hard is it to write one though? On 11/16/06, Wade Wall <wade.wall at gmail.com> wrote:> Hi all, > > I ran a discriminant analysis with some data and want to get a general idea > of prediction error rate. Some have suggested using X-fold cross validation > procedure. Anyone know if there is a function for this in R? > > Thanks, > > Wade > > [[alternative HTML version deleted]] > > ______________________________________________ > R-help at stat.math.ethz.ch mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. >-- WenSui Liu (http://spaces.msn.com/statcompute/blog) Senior Decision Support Analyst Cincinnati Children Hospital Medical Center
Weiwei Shi
2006-Nov-16 16:45 UTC
[R] X-fold cross validation function for discriminant analysis
Hi, Wade:
Some functions in R have n-fold cv themselves. For example, if you are
looking for a linear discriminant analysis (lda {MASS}), it comes with
a "leave-one-out" cv in which n equals the size of training sample,
and it gives you pretty good estimation of error rate. But be advised,
this estimated error still comes from training data, which sometimes
(in case of overfitting or condition differences b/w training and
prediction) are much bigger than the error rates in your future data.
BTW, if you are using randomForest, then it uses a out-of-bag (OOB)
method to esitmate the prediciton error.
HTH,
Weiwei
On 11/16/06, Wade Wall <wade.wall at gmail.com>
wrote:> Hi all,
>
> I ran a discriminant analysis with some data and want to get a general idea
> of prediction error rate. Some have suggested using X-fold cross
validation
> procedure. Anyone know if there is a function for this in R?
>
> Thanks,
>
> Wade
>
> [[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
--
Weiwei Shi, Ph.D
Research Scientist
GeneGO, Inc.
"Did you always know?"
"No, I did not. But I believed..."
---Matrix III
Christos Hatzis
2006-Nov-16 16:52 UTC
[R] X-fold cross validation function for discriminant analysis
One option is the Bioconductor package MLInterfaces that provides a unified interface for several machine learning alrogirithms and methods for cross-validation etc. See the algorithms web site for details. -Christos Christos Hatzis, Ph.D. Nuvera Biosciences, Inc. 400 West Cummings Park Suite 5350 Woburn, MA 01801 Tel: 781-938-3830 www.nuverabio.com -----Original Message----- From: r-help-bounces at stat.math.ethz.ch [mailto:r-help-bounces at stat.math.ethz.ch] On Behalf Of Wensui Liu Sent: Thursday, November 16, 2006 11:37 AM To: Wade Wall Cc: r-help at stat.math.ethz.ch Subject: Re: [R] X-fold cross validation function for discriminant analysis how hard is it to write one though? On 11/16/06, Wade Wall <wade.wall at gmail.com> wrote:> Hi all, > > I ran a discriminant analysis with some data and want to get a general > idea of prediction error rate. Some have suggested using X-fold cross > validation procedure. Anyone know if there is a function for this in R? > > Thanks, > > Wade > > [[alternative HTML version deleted]] > > ______________________________________________ > R-help at stat.math.ethz.ch mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. >-- WenSui Liu (http://spaces.msn.com/statcompute/blog) Senior Decision Support Analyst Cincinnati Children Hospital Medical Center ______________________________________________ R-help at stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.