Purdue University

EAS 657

Geophysical Inverse Theory

Robert L. Nowack

Lecture 6

Spectral Decompositions and Singular Value Decompositions

We will first investigate finite dimensional mappings represented by square matrices

A:R^N R^N

An eigenvector for the operator A is a vector whose direction remains unchanged upon the operation of operator A. Thus

where is a scalar called the eigenvalue associated with u. Also, a zero eigenvalue can be associated with the equation

Au = 0

The eigenvalue, , is in general a complex scalar. u is an eigenvector if with . Thus, (A-I) must be singular (not invertible). This will occur when Det(A-I) = 0. Det(A-I) = 0 is called the characteristic equation and can be factored as

Ex) A:R² R² A =

Back solving this for the eigenvectors for A gives

We now define the eigenvectors of the adjoint operator A*, w, and the corresponding eigenvalues . Since transposing doesn’t change the determinant, the eigenvalues of A* are the complex conjugates of the eigenvalues of A. Thus

Now since

if u_i is an eigenvector of A and w_j is an eigenvector of A*, then

Thus, if , then w_j must be perpendicular to u_i. This is called the biorthogonality relation for eigenvectors of A and A* for distinct .

Theorem: Let be distinct eigenvalues of A. Then the corresponding eigenvectors of A are independent. Thus (u₁, u₂, … u_N) can be used as a basis for U. This is a major utility of eigenvectors.

For each eigenvector then,

or putting all the eigenvectors into one equation

If the u_i are all linearly independent (which is not guaranteed for repeated eigenvalues), then

and

This is called a spectral representation of the linear operator A with independent eigenvectors. Of course, if the eigenvectors are not independent, then one cannot perform the inverse of M.

Ex)

but

Thus, for this example, u₁ is not distinct from u₂and a spectral decomposition cannot be performed on this matrix.

Matrices of the form

are called Jordan Blocks and have all eigenvectors for the same. For these type of matrices, spectral decompositions using eigenvectors cannot be performed.

This type of problem, with eigenvectors used as a basis for square operators, can be extended for more general decomposition of operators using the singular value decomposition. However, eigenvector expansions are very useful for self adjoint operators where

A* = A.

In this case, . Thus, the eigenvalues for any self adjoint (Hermitian matrix) operator are real. Also, the eigenvectors of A and A* are the same. Thus, u_i = w_i. In this case, for then, u_i is perpendicular u_j. Thus, the eigenvectors are directly orthogonal and not biorthogonal.

We have only showed this for distinct eigenvalues, but in fact, for an arbitrary case of repeated eigenvalues, we can always construct an orthonormal set of eigenvectors for a Hermitian matrix. (Hermitian matrices will never reduce to Jordan Blocks.) We can, therefore, always perform spectral decompositions for self adjoint operators (for Hermitian matrices)!

Any matrix with orthonormal columns is called unitary and has the property that

M^-1 = M*

M*M = MM* = I

Thus, since M is unitary for a symmetric Hermitian matrices, then

Directly in terms of the eigenvectors for a square Hermitian Matrix A,

where denotes an outer product. This is called the spectral theorem. Thus,

This scales each u_i by and the projection of x on each u_i.

For all nondefective square matrices (no Jordan Blocks), we can define functions of matrices as

where is the ith power of the matrix A and is a function of the matrix. This corresponds to the scalar expansion for as

Since

then

In general, for a nondefective square matrix,

This is called Sylvester’s theorem.

Ex) Most linear physical systems can be written as a system of first order differential equations

The formed solution to this is e^Atwhere A is a matrix. But what is e^At? By analogy to the 1-D case

has a solution

Using Sylvester’s theorem,

Solving systems of differential equations is another major use of eigenvectors and eigenvalues.

Ex) The stress “tensor” is a matrix quantity at a point in space where one index relates to the coordinate face on which the traction acts and the second index is the traction vector component.

The traction vector acting on a plane with normal can be written

sum on j

Now we want to find the vectors such that

eigenvector problem

On these faces, only normal tractions act. are the principle stress directions and are the principle stresses. The define three orthonormal directions since is symmetric, then

where M is are the principle directions, and is a diagonal matrix of principle stresses.

Generalized Matrix Analysis

Only Hermitian matrices are guaranteed not to have problems in standard eigenvector/eigenvalue analysis and non-square matrices are excluded. Fortunately, many differential operators are self adjoint which allow this sort of symmetric matrix analysis.

To avoid problems in other cases, Lanczos (1959) devised a trick of imbedding a general non-square matrix A within a larger Hermitian matrix

S is now Hermitian and, therefore, has a complete set of eigenvectors and real eigenvalues.

If A is an MxN matrix

Then S is an (M+N) x (M+N) matrix. Thus,

Solutions to define the eigenvectors and eigenvalues for S. Since we are interested in A, we will split the eigenvectors w into parts

where is length M and is length N. Then

This can also be written

This is a coupled eigenvalue problem through A and its adjoint A*. Note that if

is an eigenvector for , then

is an eigenvector for .

The eigenvalues of S are found from

If there are 2P non-zero eigenvalues, then

Since the eigenvector set for S is orthogonal (or made orthogonal for repeated eigenvalues)

then

and

Thus, u_i is perpendicular to u_j and v_i is perpendicular to v_j where

We will normalize these to 1.

The eigenvalues, , separate the coupled eigenvalue problem into

The , i = 1, N-P, span the nullspace N(A) of A, thus

and correspondingly , i = 1, M-P, span the nullspace N(A*), thus

Since these equations are now decoupled, we can now organize the eigenvectors of S. for nonzero

where P is the rank of A and also the dimension of N^perp(A) or R(A). For the zero eigenvalues,

Thus, by playing this trick of constructing S, we have been able to span the domain and range of A by eigenvector-like orthonormal vectors.

Now lets look again at the coupled eigenvector problem

Operate on the first equation by A*, then

and the second equation by A, then

where A*A is an (NxN) matrix and AA* is an (MxM) matrix. These are ordinary eigenvalue problems for (A*A) and (AA*). Note that since

(x, A*Ay) = (A*Ax, y)

(x, AA*y) = (AA*x, y)

both A*A and AA* are self adjoint (Hermitian matrices). Thus {u}, i = 1,M, and {v}, i = 1,N, form complete eigenvector systems for the operators {AA*} and {A*A}. The number of non-zero positive eigenvalues is P where .

Define

These are the “singular values” and are always positive real numbers. We can also separate the eigenvectors into matrices.

and

where are the non-zero eigenvalues for A*A and AA*. Now let

V = [V_p, V₀]

U = [U_p, U₀]

Since the columns of V and U form complete orthonormal sets of eigenvectors for A*A and AA*, then V* = V^-1 and U* = U^-1 are unitary matrices. Thus,

UU* = U*U = I_MxM

VV* = V*V = I_NxN

For the reduced set U_p (a MxP matrix) and V_p (a NxP matrix) by orthogonality of the colums of U_p, then

U_p* U_p = I_pxp V_p* V_p = I_pxp

where U_p* is PxM, U_p is MxP, V_p* is PxN and V_p is NxP. But in general, U_pU_p* I_MxM (unless P = M, and V_pV_p* I_NxN (unless P = N).

Now we want to assemble this into one representation for A.

where A is MxN, V_p is NxP, U_p is MxP, V₀ is Nx(N-P), A* is NxM, is PxP, and U₀ is
Mx(M-P). We can then write

and

If we multiple this through, then the only non-zero term is

This is called the singular value decomposition of the operator or matrix where

The columns of U_p and V_p are orthonormal vectors that span R(A) and N^perp(A) respectively. This is a general representation of any matrix regardless of whether it is square or non-square, or whether it is complete or defective. The subspaces spanned by U₀ and V₀ can be thought of as “blindspots” not illuminated by A.

Ex) Let

v₁ + v₂ = 1

v₃ = 2

-v₃ = 1

This can be written in matrix form as