R help - Jun 2004 - How to determine the number of dominant eigenvalues in PCA

Dear All,

I want to know if there is some easy and reliable way
to estimate the number of dominant eigenvalues
when applying PCA on sample covariance matrix.

Assume x-axis is the number of eigenvalues (1, 2, ....,n), and y-axis is the 
corresponding eigenvalues (a1,a2,..., an) arranged in desceding order.
So this x-y plot will be a decreasing curve. Someone mentioned using the elbow
(knee) method
to find the point that the maximal curvature of this curve occurs.
The number at this point would be the number of dominant eigenvalues.

But I could not find any reference papers on this idea.
Does anyone has tried this method or knows more details on this?

Thanks for your point.

Fred

	[[alternative HTML version deleted]]

On Mon, 28 Jun 2004, Fred wrote:
> I want to know if there is some easy and reliable way
> to estimate the number of dominant eigenvalues
> when applying PCA on sample covariance matrix.
The short answer is `no' since it depends what you want to do PCA for (and 
there are many possible uses).
> Assume x-axis is the number of eigenvalues (1, 2, ....,n), and y-axis is
the
> corresponding eigenvalues (a1,a2,..., an) arranged in desceding order.
> So this x-y plot will be a decreasing curve. Someone mentioned using the
elbow (knee) method
> to find the point that the maximal curvature of this curve occurs.
> The number at this point would be the number of dominant eigenvalues.
It's not a curve!  If you joins the points by line it is piecewise linear 
and has curvature nowhere.

See ?screeplot and its references, since the plot is called a `scree
plot'.  It's a well known technique in all good textbooks on PCA.
> But I could not find any reference papers on this idea.
> Does anyone has tried this method or knows more details on this?
-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

Fred wrote:
> Dear All,
> 
> I want to know if there is some easy and reliable way to estimate the
> number of dominant eigenvalues when applying PCA on sample covariance
> matrix.
> 
> Assume x-axis is the number of eigenvalues (1, 2, ....,n), and y-axis
> is the corresponding eigenvalues (a1,a2,..., an) arranged in
> desceding order. So this x-y plot will be a decreasing curve. Someone
> mentioned using the elbow (knee) method to find the point that the
> maximal curvature of this curve occurs. The number at this point
> would be the number of dominant eigenvalues.
> 
> But I could not find any reference papers on this idea. Does anyone
> has tried this method or knows more details on this?
> 
> Thanks for your point.
> 
> Fred
> 
Try this reference from the field of ecology:

@Article{571,
   Author         = {D. A. Jackson},
   Title          = {Stopping rules in principal components analysis: a
                    comparison of heuristic and statistical approaches},
   Journal        = {Ecology},
   Volume         = {74},
   Number         = {8},
   Pages          = {2204--2214},
   month          = {},
   year           = 1993
}

Gav
-- 
%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%
Gavin Simpson                     [T] +44 (0)20 7679 5522
ENSIS Research Fellow             [F] +44 (0)20 7679 7565
ENSIS Ltd. & ECRC                 [E] gavin.simpson at ucl.ac.uk
UCL Department of Geography       [W] http://www.ucl.ac.uk/~ucfagls/cv/
26 Bedford Way                    [W] http://www.ucl.ac.uk/~ucfagls/
London.  WC1H 0AP.
%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%

Hi!

There is a chapter in the book from H佷rdl about the interpretation of PCs
available online.
http://www.quantlet.com/mdstat/scripts/mva/htmlbook/mvahtmlframe93.html


About determining the number of dominant eigenvalues is a chapter in book of A.
Handl  (available online but in german.)
http://www.quantlet.com/mdstat/scripts/mst/html/msthtmlframe56.html

Two references to this topic from this online book.
Cattell, R. B. (1966): The scree test for the number of factors. Multivariate
Behavioral Research, 1, 245-276
Kaiser, H. F. (1960): The application of electronic computers to factor
analysis. Educ. Psychol. Meas., 20, 141-151



Hope this helps.
Sincerely Eryk



*********** REPLY SEPARATOR  ***********

On 28.06.2004 at 10:06 Fred wrote:
>Dear All,
>
>I want to know if there is some easy and reliable way
>to estimate the number of dominant eigenvalues
>when applying PCA on sample covariance matrix.
>
>Assume x-axis is the number of eigenvalues (1, 2, ....,n), and y-axis is
>the 
>corresponding eigenvalues (a1,a2,..., an) arranged in desceding order.
>So this x-y plot will be a decreasing curve. Someone mentioned using the
>elbow (knee) method
>to find the point that the maximal curvature of this curve occurs.
>The number at this point would be the number of dominant eigenvalues.
>
>But I could not find any reference papers on this idea.
>Does anyone has tried this method or knows more details on this?
>
>Thanks for your point.
>
>Fred
>
>	[[alternative HTML version deleted]]
>
>______________________________________________
>R-help at stat.math.ethz.ch mailing list
>https://www.stat.math.ethz.ch/mailman/listinfo/r-help
>PLEASE do read the posting guide!
http://www.R-project.org/posting-guide.html