Lecture 17

[[lecture-data]]

2024-10-07

Caution

Exam on the 15th from 7-10pm

4. Chapter 4

The main theorem of this chapter is Courant-Fisher Theorem

Courant-Fisher Theorem

let $A \in M_{n}$ be hermitian with eigenvalues $λ_{1} \leq \dots \leq λ_{n}$ (since $A$ is hermitian, its eigenvalues are real (see this theorem)). Then $\forall k = 1, 2, \dots n$ we have

max_{y_{1}, y_{2}, \dots, y_{k - 1} \in C^{n}} {min_{x \in C^{n} \neq 0, x ⊥ y_{1}, y_{2}, \dots, y_{k + 1}} \frac{x^{*} A x}{x^{*} x}} = λ_{k}

and

min_{y_{k + 1}, \dots, y_{n} \in C^{n}} {max_{x \in C^{n} \neq 0, x ⊥ y_{k + 1}, \dots, y_{n}} \frac{x^{*} A x}{x^{*} x}} = λ_{k}

Claim

Consider ${a_{i}}_{i \in I}$ and ${b_{i}}_{i \in I}$ . Suppose for all $i$ , we have that $a_{i} \leq b_{i}$ . Then

min_{i \in I} a_{i} \leq min_{i \in I} b_{i}

and

max_{i \in I} a_{i} \leq max_{i \in I} b_{i}

Further, if I minimize (or maximize) over some conditions $κ$ , these inequalities still hold provided I apply the same conditions to both sides. I can even make one of these conditions a minimization (or maximization) over some more conditions!

We also have for conditions $κ$ and additional conditions $κ^{'}$ that
$min_{κ} x y z \leq min_{κ \cup κ^{'}} x y z$
("the minimization might not work as well")

And so
$max min_{κ} x y z \leq max min_{κ \cup κ^{'}} x y z$

Weyl's Theorem

For any $A, B \in M_{n}$ hermitian, for any $k = 1, 2, \dots, n$ we have

λ_{k} (A) + λ_{1} (B) \leq λ_{k} (A + B) \leq λ_{k} (A) + λ_{n} (B)

(where $λ_{i}$ is the $i$ th largest eigenvalue)

Proof

$\begin{aligned} λ_{k} (A + B) & = min max_{courant-fisher conditions} \frac{x^{*} (A + B) x}{x^{*} x} \\ = min max_{C F} \frac{x^{*} A x}{x^{*} x} + \frac{x^{*} B x}{x^{*} x} \\ = min max_{C F} \frac{x^{*} A x}{x^{*} x} + λ_{n} (B) (*) \\ = λ_{n} (B) + λ_{k} (A) (* *) \end{aligned}$

Where $(*)$ is due to Rayleigh-Ritz and $(* *)$ is due to Courant-Fisher. The other inequality follows analogously

(see Weyl's Theorem)

Corollary

If $A, B \in M_{n}$ hermitian and $B$ is postive semidefinite, then for all $k = 1, 2, \dots, n$ we have

λ_{k} (A) \leq λ_{k} (A + B)

Proof

Since $λ_{1} (B) > 0$ , by Weyl $λ_{k} (A) \leq λ_{k} (A) + λ_{1} (B) \leq λ_{k} (A + B)$

Interlacing Theorem I

If $A \in M_{n}$ is hermitian, $a \in R$ , $z \in C^{n}$ , then for all $k$ :

λ_{k} (A + a z z^{*}) \leq λ_{k + 1} (A)

Proof

$\begin{aligned} λ_{k} (A + a z z^{*}) & = max_{y_{1}, y_{2}, \dots, y_{k - 1} \in C^{n}} min_{x \in C^{n} \neq 0, x ⊥ y_{1}, \dots, y_{k - 1}} \frac{x^{*} (A + a z z^{*}) x^{*}}{x^{*} x} \\ \leq max_{y_{1}, \dots, y_{k - 1} \in C^{n}} min_{x \in C^{n} \neq 0, x ⊥ y_{1}, \dots, y_{k - 1}, x ⊥ z} \frac{x^{*} (A + a z z^{*}) x^{*}}{x^{*} x} \\ = max_{y_{1}, \dots, y_{k - 1} \in C^{n}} min_{x \in C^{n} \neq 0, x ⊥ y_{1}, \dots, y_{k - 1}, z} \frac{x^{*} A x^{*}}{x^{*} x} (*) \\ \leq max_{y_{1}, y_{2}, \dots, y_{k} \in C^{n}} min_{x \in C^{n} \neq 0, x ⊥ y_{1}, \dots, y_{k}} \frac{x^{*} A x^{*}}{x^{*} x} (* *) \\ = λ_{k + 1} (A) by Courant-Fisher \end{aligned}$

$(*)$ since if $z ⊥ x$ , then $a x^{*} z z^{*} x = 0$
$(* *)$ since we can choose $y_{k} = z$

(see interlacing theorem 1)

Theorem

In general, if $B \in M_{n}$ is hermitian, then it is unitarily diagonalizable, say $B = U D U^{*}, U = [u_{1} | u_{2} | \dots | u_{n}]$ and $λ_{1}, λ_{2}, \dots, λ_{n}$ the diagonal elements of $D$ . Then

B = U D U^{*} = \sum_{i = 1}^{n} λ_{i} u_{i} u_{i}^{*}

(see normal matrices are the sum of rank-1 matrices)

Corollary

Let $A, B \in M_{n}$ be hermitian and rank of $B$ is $r$ , then for all $k$ ,

λ_{k} (A + B) \leq λ_{k + r} (A)

Proof

This follows directly from the interlacing theorem 1 and the fact that hermitian matrices are the sum of rank-1 matrices

Say $B = U D U^{*}, U = [u_{1} | u_{2} | \dots | u_{n}]$ unitary. WLOG let the entries $d_{11}, d_{22}, \dots, d_{r r}$ be the only non-zero eigenvalues / nonzero entries of $D$ (we can do this because $B$ is diagonalizable). Then

\begin{aligned} λ_{k + r} (A) & \geq λ_{k + r - 1} (A + d_{r r} u_{r} u_{r}^{*}) (*) \\ \geq λ_{k + r - 2} (A + d_{r r} u_{r} u_{r}^{*} + d_{(r - 1) (r - 1)} u_{r - 1} u_{r - 1}^{*}) \\ \geq \dots \\ \geq λ_{k} (A + \sum_{i = 1}^{r} d_{i i} u_{i} u_{i}^{*}) \\ = λ_{k} (A + B) (* *) \end{aligned}

Where we get $(*)$ from the interlacing theorem 1 and $(* *)$ from hermitian matrices are the sum of rank-1 matrices

Corollary

Let $A, B \in M_{n}$ be hermitian and $B$ is rank $r$ . Then for all $k$

λ_{k - r} (A + B) \leq λ_{k} (A) \leq λ_{k + r} (A + B)

Proof

This is a direct result/generalization of the above corollary.

λ_{k - r} (A) \leq λ_{k} (A + B) \leq λ_{k + r} (A)

Let $A + B = A$ and $- B = B$ . Then from the above we have

λ_{k} (A + B) \leq λ_{k + r} (A)

(see bounds on eigenvalues for sums of hermitian matrices)