On 29.04.2010 15:01, Eric Elguero wrote:> Hi,
>
> a colleague ran a stepwise discriminant analysis
> twice in a row and got different results, suggesting
> some "sochasticity" in the algorithms involved.
> I looked at her data and found that there was a lot
> of collinearity, so that I reckoned that maybe "stepclass"
> (klaR) cannot find a clear winner when trying to include a
> new variable and makes a random choice. Is that true?
Yes, since a cross validation is involved.
If you want stable results, you could try leave one out or set a seed.
Anyway, if you variables are collinear I wonder if the stepwise approach
is the smartest solution here.....
> another possibility is that "lda" (from MASS) computes
> CV classification rates from a random subsample instead of
> using all the data (?) That might be a sensible choice
> with a very large sample.
> I advised her to run the function several times and
> see if a consensus emerges, but that doesn't seem to
> be the case, and besides, I would like to know what
> really is going on.
Well, it is called cross validation which is based on random sampling if
you do not have k=n -fold CV (=leave-one-out).
Again, to get reproducible results, you will need to set a seed.
If the results are that unstable: Do you really have a sufficient number
of observations for your classification problem?
Uwe Ligges
>
> thanks
>
> Eric Elguero
> Laboratory Genetics and Evolution of Infectious Diseases,
> Team: Genetics and Adaptation of Plasmodium
> UMR 2724 CNRS-IRD,
> IRD Montpellier,
> 911 Avenue Agropolis, BP 64501,
> 34394 Montpellier Cedex 5,
> France
>
>
>> f4.U.spDA<- stepclass(f.mes, f.gp4,
> "lda",improvement=0.01,prior=rep(0.25,4))
> `stepwise classification', using 10-fold cross-validated correctness
> rate of method lda'.
> 89 observations of 31 variables in 4 classes; direction: both
> stop criterion: improvement less than 1%.
> correctness rate: 0.58333; in: "X2"; variables (1): X2
> correctness rate: 0.66389; in: "X9"; variables (2): X2, X9
> correctness rate: 0.69583; in: "X27"; variables (3): X2, X9,
X27
>
> hr.elapsed min.elapsed sec.elapsed
> 0.00 0.00 20.77
>
>> f4.U.spDA<- stepclass(f.mes, f.gp4,
> "lda",improvement=0.01,prior=rep(0.25,4))
> `stepwise classification', using 10-fold cross-validated correctness
> rate of method lda'.
> 89 observations of 31 variables in 4 classes; direction: both
> stop criterion: improvement less than 1%.
> correctness rate: 0.60556; in: "X2"; variables (1): X2
> correctness rate: 0.71806; in: "X6"; variables (2): X2, X6
>
> hr.elapsed min.elapsed sec.elapsed
> 0.00 0.00 15.14
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.