Mo logo [home] [lexicon] [problems] [tests] [courses] [auxiliaries] [notes] [staff] german flag
Mathematics-Online course: Linear Algebra - Normal Forms - Singular Value Decomposition

Singular Value Decomposition

[previous page] [next page] [table of contents][page overview]
For any real $ m\times n$ matrix $ A$ there exist unitary matrices $ U$ and $ V$ with

$\displaystyle U^* A V = S = \left(\begin{array}{ccc} s_1 & & 0 \\ & s_2 & \\ 0 & & \ddots \end{array}\right). $

The singular values

$\displaystyle s_1\ge s_2\ge\cdots\ge s_k>s_{k+1}=\cdots=0 $

are the square roots of the eigenvalues of $ A^* A$, $ k$ is the rank of $ A$, and the columns of $ U$ and $ V$ are eigenvectors of $ AA^*$ and $ A^* A$, respectively. Moreover,

$\displaystyle Av_j = \sigma_j u_j $

for $ 1\leq j \leq k$.

In the special case of a real matrix $ A$, the matrices $ U$ and $ V$ are orthogonal.

Diagonalizing the hermitian matrix $ A^* A$ we obtain

$\displaystyle V^* A^* A V = \operatorname{diag}(s_1^2,\ldots,s_k^2,0,\ldots,0) = S^{\operatorname t}S\, ,$ (1)

and we can assume that, beginning with the top left entry, the eigenvalues are arranged in descending order. Since $ A^* A$ is positive semi-definite, all eigenvalues are non-negative and can be interpreted as squares of non-negative real numbers which are used to define the $ m\times n$ diagonal matrix $ S$ .

From the above equation, it follows that the columns of $ AV$ are orthogonal and have norm $ s_i$ :

$\displaystyle AV = \left(\begin{array}{ccc ccc} s_1 u_1 & \cdots & s_k u_k & 0 & \cdots & 0 \end{array}\right) = US $

with a unitary matrix $ U$ .

Having constructed the representation

$\displaystyle A= USV^* \ , $

we can easily verify the remaining assertions.

Since unitary transformations do not change the rank of a matrix, we have

$\displaystyle \operatorname{rank}\,A = \operatorname{rank}\,S = k\,. $

Moreover, we have

$\displaystyle AA^* u_j = U(SS^{\operatorname t})U^* u_j = u_j (s_j)^2 $

and

$\displaystyle A^*Av_j = V(S^{\operatorname t}S) V^* v_j = v_j (s_j)^2\,. $

Finally, $ Av_j$ equals the $ j$ -th column of $ US$ , i. e. $ u_js_j$ .

  automatically generated 4/21/2005