R help - May 2004 - stability measures for heirarchical clustering

Dear R users,
 
I'm interested in measuring the stability of a heirarchical clustering, of
the overall clustering and finding sub clusters (from cutting the
heirarchical clustering at different levels) which demonstrate stability. 
I saw some postings on the R help from a while back about bootstrapping for
clustering (using sample and generating a consesus tree with a web based
tool CONSENSE) but i wondered if there have been any advances on the
"bootstrapping clustering" front? 
In terms of finding stability in sub sections of the clustering I'm thinking
of modifying the jaccard function from prabclus to look at pairwise
similarities in different cluster partitions of sub-samples of the data,
with high similarity being indicative of stability. 
I wondered if anyone has already looked at stability measures for clustering
(particularly thos which interface with hclust), and if any are available
already in R but i have just missed them?
 
I realise there are problems with heirarchical clustering..and i may have to
consider using a different method,
 
thanks for your advice,
 
Jacqueline Hall

	[[alternative HTML version deleted]]

Dear Jacqueline,

may be the corrected rand index implemented in cluster.stats, package fpc,
and the literature on its help page may be of interest to you. As far as I
know, the corrected rand is the index cited most often for comparing
different clusterings on the same points. This can be used together with
bootstrapping, for example. Perhaps something reasonable can
also be done with jaccard; I am not sure.

Best,
Christian

PS:
On Tue, 11 May 2004, Jacqueline Hall wrote:
> Dear R users,
>  
> I'm interested in measuring the stability of a heirarchical clustering,
of
> the overall clustering and finding sub clusters (from cutting the
> heirarchical clustering at different levels) which demonstrate stability. 
> I saw some postings on the R help from a while back about bootstrapping for
> clustering (using sample and generating a consesus tree with a web based
> tool CONSENSE) but i wondered if there have been any advances on the
> "bootstrapping clustering" front? 
> In terms of finding stability in sub sections of the clustering I'm
thinking
> of modifying the jaccard function from prabclus to look at pairwise
> similarities in different cluster partitions of sub-samples of the data,
> with high similarity being indicative of stability. 
> I wondered if anyone has already looked at stability measures for
clustering
> (particularly thos which interface with hclust), and if any are available
> already in R but i have just missed them?
>  
> I realise there are problems with heirarchical clustering..and i may have
to
> consider using a different method,
...there are often good reasons for hierarchical clustering, so if
you know what you are doing, do not let the others confuse you..


***********************************************************************
Christian Hennig
Fachbereich Mathematik-SPST/ZMS, Universitaet Hamburg
hennig at math.uni-hamburg.de, http://www.math.uni-hamburg.de/home/hennig/
#######################################################################
ich empfehle www.boag-online.de