R help - Jun 2007 - Spectral Decomposition

All of my resources for numerical analysis show that the spectral
decomposition is

A = CBC'

Where C are the eigenvectors and B is a diagonal matrix of eigen values.
Now, using the eigen function in R

# Original matrix
aa <- matrix(c(1,-1,-1,1), ncol=2)

ss <- eigen(aa)

# This results yields back the original matrix according to the formula
above
ss$vectors %*% diag(ss$values) %*% t(ss$vectors)

However, for the following I do not get back the original matrix using
the general formula for the spectral decomposition:

set.seed(123)

aa <- matrix(rnorm(16), ncol=4)

ss <- eigen(aa)

ss$vectors %*% diag(ss$values) %*% t(ss$vectors)

However, if I do the following

A = CBC^{-1}

I get back the original matrix A

ss$vectors %*% diag(ss$values) %*% solve(ss$vectors)

In my lit search I am not finding an explanation for why this works, so
I am seeking R-help since there may be a computational rationale that
can be explained by users (or authors) of the function. In my
experimentation with some computations it seems that the latter approach
is more general in that it yields back the matrix I began with, but
deviates from the formula I commonly see in texts.

Thanks,
Harold
> sessionInfo()
R version 2.5.0 (2007-04-23) 
i386-pc-mingw32 

locale:
LC_COLLATE=English_United States.1252;LC_CTYPE=English_United
States.1252;LC_MONETARY=English_United
States.1252;LC_NUMERIC=C;LC_TIME=English_United States.1252

attached base packages:
[1] "stats"     "graphics"  "grDevices"
"utils"     "datasets"
"methods"   "base"     

other attached packages:
 MiscPsycho     statmod      mlmRev        lme4      Matrix     lattice 
      "1.0"     "1.3.0"   "0.995-1"
"0.99875-2" "0.99875-3"    "0.15-4"


	[[alternative HTML version deleted]]

For a general square matrix A, the eigenvalue decomposition is
A = CBC^{-1}
For the special case of symmetric A,
C^{-1} = C'

On 29-Jun-07 12:29:31, Doran, Harold wrote:
> All of my resources for numerical analysis show that the spectral
> decomposition is
> 
> A = CBC'
> 
> Where C are the eigenvectors and B is a diagonal matrix of eigen
> values. Now, using the eigen function in R
> 
># Original matrix
> aa <- matrix(c(1,-1,-1,1), ncol=2)
> 
> ss <- eigen(aa)
> 
># This results yields back the original matrix according to the
> formula above
> ss$vectors %*% diag(ss$values) %*% t(ss$vectors)
> 
> However, for the following I do not get back the original matrix
> using the general formula for the spectral decomposition:
> 
> set.seed(123)
> 
> aa <- matrix(rnorm(16), ncol=4)
> 
> ss <- eigen(aa)
> 
> ss$vectors %*% diag(ss$values) %*% t(ss$vectors)
> 
> However, if I do the following
> 
> A = CBC^{-1}
> 
> I get back the original matrix A
> 
> ss$vectors %*% diag(ss$values) %*% solve(ss$vectors)
Harold, I think the key to the issue is whether your original
matric is symmetric or not. For your formula

  A = C*B*C'

to work, where B is a diagonal matrix (therefore essentially
symmetric) you have -- bearing in mind the reversal of factors --

  A' = ReverseFactorsIn(C'*B'*C) = C*B*C' = A

so A would have to be symmetric. This was the case for your
first example matrix(c(1,-1,-1,1), ncol=2).

However, your second example will not be symmetric, so the
formula will not work, and you will need A = C*B*C^(-1).

If A is not symmetric, you have "left" eigenvectors:

  x'*A = lambda*x'

and "right" eigenvectors:

  A*x = lambda*x

and the "left" eigenvectors are not the same as the "right"
eigenvectors, though you have the same set of eigenvalues lambda
in each case.

You then have

  A = L'*B*R

Of course the most frequent occurrence of this kind of question
in statistics is where A is a covariance or correlation matrix,
which is symmetric by definition.

Hoping this helps!
Ted.
> In my lit search I am not finding an explanation for why this works, so
> I am seeking R-help since there may be a computational rationale that
> can be explained by users (or authors) of the function. In my
> experimentation with some computations it seems that the latter
> approach is more general in that it yields back the matrix I began
> with, but deviates from the formula I commonly see in texts.
> 
> Thanks,
> Harold
--------------------------------------------------------------------
E-Mail: (Ted Harding) <ted.harding at nessie.mcc.ac.uk>
Fax-to-email: +44 (0)870 094 0861
Date: 29-Jun-07                                       Time: 14:23:03
------------------------------ XFMail ------------------------------