R help - Mar 2006 - clustering a sparse dissimilarity matrix

Dear all,

I have a BIG dissimilarity matrix (around one thousand by one thousand), 
that I would like to cluster.  Most of the elements of the matrix are 
zero or close to zero.  Is there a way to cluster the matrix 
(hierarchical or partitioning methods) that discards those elements that 
are close to zero (by using a specified threshold on the matrix)? I am 
asking this because otherwise I get a huge amount of clutter for 
singletons or very small clusters.  Also, how can you look for clusters 
of a specified  size, apart from looking visually at the dendrogram?  Is 
there a way to bias the algorithm specifically for clusters of a certain 
size?

thank you very much for any suggestion

best regards

   giuseppe

PS
Note that I cannot use the original data instead of the dissimilarity 
matrix because those are dissimilarities (computed from the spatial 
correlation coefficient) between fMRI brain maps, each of which has 
around 60000 variables.

-- 
---------------------------------
Giuseppe Pagnoni
Psychiatry and Behavioral Sciences
Emory University School of Medicine
1639 Pierce Drive, Suite 4000
Atlanta, GA, 30322
tel: 404.712.8431
fax: 404.727.3233

Guiseppe,

there are several things unclear with this question.

On Thu, 30 Mar 2006, Giuseppe Pagnoni wrote:
> Dear all,
>
> I have a BIG dissimilarity matrix (around one thousand by one thousand),
> that I would like to cluster.  Most of the elements of the matrix are
> zero or close to zero.  Is there a way to cluster the matrix
> (hierarchical or partitioning methods) that discards those elements that
> are close to zero (by using a specified threshold on the matrix)?
What do you mean by an "element close to zero"?
An entry in the matrix? Then, what do you mean by "discarding" them?
How
should that help?
> I am
> asking this because otherwise I get a huge amount of clutter for
> singletons or very small clusters.  Also, how can you look for clusters
> of a specified  size, apart from looking visually at the dendrogram?  Is
> there a way to bias the algorithm specifically for clusters of a certain
> size?
What clustering method did you apply and why?
How would you like to specify the size of your clusters (and why)?

Best,
Christian


*** --- ***
Christian Hennig
University College London, Department of Statistical Science
Gower St., London WC1E 6BT, phone +44 207 679 1698
chrish at stats.ucl.ac.uk, www.homepages.ucl.ac.uk/~ucakche