R help - Apr 2007 - heatmap and phylogram / dendogram ploting problem, ape package

I am having trouble displaying a dendrogram of evolutionary
relationships (a phylogram imported from the ape package) as the
vertical component of a heatmap, but keeping the hierarchical
clustering of the horizontal component.  The relationships of the
vertical component in the generated heatmap are not that of the
dendrogram, although the ordering is.

In more detail, I am attempting to generate a heatmap from a table
that contains the abundance of different bacteria at different
locations, with a dendrogram that groups the
environments by the pattern of bacterial abundance.  This is easy, thanks to
a nice code snippet at the R Graph Gallery
(http://addictedtor.free.fr/graphiques/RGraphGallery.php?graph=66):

env <- read.table("env.dat")
x  <- as.matrix(env)
hv <- heatmap(x, col = cm.colors(256), scale="none",
              xlab = "Environments", ylab= "Species",
              main = "heatmap of species present in environments")

However, instead of a dendrogram that groups the rows (vertical axis)
by the abundance pattern (as above), I would like to force it to order
and display a dendrogram
indicating their evolutionary relatedness.  I am using the excellent ape
package (http://cran.r-project.org/src/contrib/Descriptions/ape.html) to
import the evolutionary dendrograms.  I have already manipulated the
dendrogram to be ultrameric, with branches all the same length, to
prevent an error, although I would prefer not to have to do so:

library(ape)
mytree <- read.tree(file = "ultra.newick", text = NULL, tree.names
NULL, skip = 0, comment.char = "#")
#I then change them into a hclust:
tree <- as.hclust(mytree)
#and make this into a dendrogram
dend <- as.dendrogram(tree)

However, when I use this dendrogram as part of the heatmap, the relationships
in the dendrogram that I'm loading are not preserved, although the order of
bacteria in the heatmap changes:

hv <- heatmap(x, col = cm.colors(256), scale="none",
        Rowv=dend, keep.dendro=TRUE,
        xlab = "Environments", ylab= "Species",
        main = "heatmap of species present in environments")

Is there something obvious I am missing?  When I plot the hclust and
dendrogram, they seem to preserve the relationships that I am attempting to
show, but not when the dendrogram is used in the heatmap.  I'm not sure I
really understand the datatype of R dendrogram objects, and this may
be the source of my
problems.  The heatmap documentation
(http://addictedtor.free.fr/graphiques/help/index.html?pack=stats&alias=heatmap&fun=heatmap.html)
is somewhat opaque to someone with my programing skills.  Would it be
better to reorder the heatmap and then somehow add in the dendrogram
with add.expr command?

If anyone could suggest something, or point me in the right direction I would
greatly appreciate it.


   jamesh

Here are the contents of the two files used as examples above:

env.dat:
soil1 soil2 soil3 marine1 marine2 marine3
One 23 24 26 0 0 0
Two 43 43 43 3 6 8
Three 56 78 45 23 45 56
Four 6 6 3 34 56 34
Five 2 17 12 76 45 65




ultra.newick:
(((One:1,Four:1):1,(Three:1,Two:1):1):1,Five:3):0.0;

Emmanuel Paradis, the maintainer of the ape package was very helpful
in solving this problem.  It seems that it heatmap does not reorder
the rows, so you must reorder them or change the heatmap code to do
so.  The heatmap maintainers will document this, but not change the
behavior.  The following fix works for me:
> On 4/30/07,  Emmanuel wrote:
> . . . Thanks for your interesting case study. I found out that, apparently,
> your problem comes from a bug in heatmap: it assumes that the row names
> of the matrix are in the same order than the labels of the dendrogram.
> In your case, the following fix seems to work. You have to edit heatmap
> with fix(heatmap), find the line:
>
>     x <- x[rowInd, colInd]
>
> and change it for:
>
>     x <- x[labels(ddr)[rowInd], colInd]
>
> I think this will not solve the problem in all cases. . .
> On 4/24/07, jamesrh <jamesrh at gmail.com> wrote:
> > I am having trouble displaying a dendrogram of evolutionary
> > relationships (a phylogram imported from the ape package) as the
> > vertical component of a heatmap, but keeping the hierarchical
> > clustering of the horizontal component.  The relationships of the
> > vertical component in the generated heatmap are not that of the
> > dendrogram, although the ordering is.
> >
> > In more detail, I am attempting to generate a heatmap from a table
> > that contains the abundance of different bacteria at different
> > locations, with a dendrogram that groups the
> > environments by the pattern of bacterial abundance.  This is easy,
thanks
> > to a nice code snippet at the R Graph Gallery
> > (http://addictedtor.free.fr/graphiques/RGraphGallery.php?graph=66):
> >
> > env <- read.table("env.dat")
> > x  <- as.matrix(env)
> > hv <- heatmap(x, col = cm.colors(256), scale="none",
> >               xlab = "Environments", ylab=
"Species",
> >               main = "heatmap of species present in
environments")
> >
> > However, instead of a dendrogram that groups the rows (vertical axis)
> > by the abundance pattern (as above), I would like to force it to order
> > and display a dendrogram
> > indicating their evolutionary relatedness.  I am using the excellent
ape
> > package (http://cran.r-project.org/src/contrib/Descriptions/ape.html)
to
> > import the evolutionary dendrograms.  I have already manipulated the
> > dendrogram to be ultrameric, with branches all the same length, to
> > prevent an error, although I would prefer not to have to do so:
> >
> > library(ape)
> > mytree <- read.tree(file = "ultra.newick", text = NULL,
tree.names > > NULL, skip = 0, comment.char = "#")
> > #I then change them into a hclust:
> > tree <- as.hclust(mytree)
> > #and make this into a dendrogram
> > dend <- as.dendrogram(tree)
> >
> > However, when I use this dendrogram as part of the heatmap, the
> > relationships in the dendrogram that I'm loading are not
preserved,
> > although the order of bacteria in the heatmap changes:
> >
> > hv <- heatmap(x, col = cm.colors(256), scale="none",
> >         Rowv=dend, keep.dendro=TRUE,
> >         xlab = "Environments", ylab= "Species",
> >         main = "heatmap of species present in environments")
> >
> > Is there something obvious I am missing?  When I plot the hclust and
> > dendrogram, they seem to preserve the relationships that I am
attempting
> > to show, but not when the dendrogram is used in the heatmap.  I'm
not
> > sure I really understand the datatype of R dendrogram objects, and
this
> > may be the source of my
> > problems.  The heatmap documentation
> >
(http://addictedtor.free.fr/graphiques/help/index.html?pack=stats&alias=h
> >eatmap&fun=heatmap.html) is somewhat opaque to someone with my
programing
> > skills.  Would it be better to reorder the heatmap and then somehow
add
> > in the dendrogram with add.expr command?
> >
> > If anyone could suggest something, or point me in the right direction
I
> > would greatly appreciate it.
> >
> >
> >    jamesh
> >
> > Here are the contents of the two files used as examples above:
> >
> > env.dat:
> > soil1 soil2 soil3 marine1 marine2 marine3
> > One 23 24 26 0 0 0
> > Two 43 43 43 3 6 8
> > Three 56 78 45 23 45 56
> > Four 6 6 3 34 56 34
> > Five 2 17 12 76 45 65
> >
> >
> >
> >
> > ultra.newick:
> > (((One:1,Four:1):1,(Three:1,Two:1):1):1,Five:3):0.0;