Lecture 29

[[lecture-data]]

2024-11-08

Readings

5. Chapter 5

recall theorem from last time that for complete NLS, we have absolute convergence implies convergence

Theorem

Suppose that $B \in M_{n}$ and $| | \cdot | |$ is a matrix norm on $M_{n}$ such that $| | B | | < 1$ . Then $I - B$ is invertible and $(I - B)^{- 1} = \sum_{i = 1}^{\infty} B^{i}$ .

This follows from absolute convergence implies convergence in complete NLS.

Proof

$\begin{aligned} (I - B) \sum_{i = 1}^{\infty} B^{i} & = [I + B + B^{2} + B^{3} + \dots] + [- B - B^{2} - B^{3} - B^{4} - \dots] \\ = I - B^{N + 1}, N \to \infty \\ | | B | | < 1 & ⟹ | | B^{N + 1} | | \leq | | B | |^{N + 1} \to 0 \\ ⟹ | | I - (I - B) \sum_{i = 1}^{\infty} B^{i} | | & = | | B^{N + 1} | | \to 0 \\ ⟹ (I - B) \sum_{i = 1}^{\infty} B^{i} & = I \\ ⟹ \sum_{i = 1}^{\infty} B^{i} = (I - B)^{- 1} \end{aligned}$

(see matrices with norm less than 1 define an infinite series inverse)

Note

If $B \in M_{n}$ where $ρ (B) < 1$ , then $I - B$ is invertible and the inverse is as claimed.

Proof

This follows immediately from this theorem and we can find a matrix norm that evaluates arbitrarily close to the spectral radius. (so if $ρ < 1$ , we can find a matrix norm that is less than 1)

(see matrices with spectral radius less than 1 define an infinite series inverse)

Theorem

If $A \in M_{n}$ such that $| | I - A | | < 1$ , then $A$ is invertible and $A^{- 1} = \sum_{i = 1}^{\infty} (I - A)^{i}$

Proof

This follows immediately from matrices with norm less than 1 define an infinite series inverse by just defining $A = I - B$

Compatible Norm

A matrix norm $| | \cdot | |$ on $M_{n}$ is compatible with vector norm $| | \cdot | |^{'}$ on $C^{n}$ precisely when for all $A \in M_{n}, x \in C^{n}$ we have

| | A x | |^{'} \leq | | A | | \cdot | | x | |^{'}

(if $| | \cdot | |$ is induced by $| | \cdot | |^{'}$ , then by definition they are compatible!!)

(see compatible norm)

Def

Suppose $| | \cdot | |$ is a matrix norm on $M_{n}$ . The condition number for any (invertible) matrix $A \in M_{n}$ is defined as

κ_{| | \cdot | |} (A) := | | A | | \cdot | | A^{- 1} | |

Notes

For any $A \in M_{n}$ invertible (and $| | \cdot | |$ matrix norm on $M_{n}$ ), $κ (A) \geq 1$ . - -

Since $| | I | | \leq | | I | | \cdot | | I | | ⟹ 1 \leq | | I | |$ , we get that $κ (A) | | A | | \cdot | | A^{- 1} | | \geq (*) | | A A^{- 1} | | = | | I | | \geq 1$ .
$(*)$ is due to submultiplicity

(see condition number (matrix analysis))

Theorem

Suppose $A \in M_{n}$ is invertible and $b, x, Δ b, Δ x \in C^{n}$ such that $b, x \neq 0$ . And suppose $| | \cdot | |$ is a matrix norm on $M_{n}$ is compatible with $| | \cdot | |^{'}$ on $C^{n}$ . Further, suppose that $A x = b$ and $A (x + Δ x) = b + Δ b$ .

(this is a "noisy" linear system in the LHS observations and the RHS solution)

\frac{1}{κ (A)} \frac{| | Δ b | |}{| | b | |} \leq \frac{| | Δ x | |}{| | x | |} \leq κ (A) \frac{| | Δ b | |}{| | b | |}

(see condition number determines the amount of noise in a linear system)

Proof

Subtracting the above yields $A (Δ x) = Δ b$ . So

By compatibility, $| | B | | \leq | | A | | \cdot | | x | |$
$⟹ \frac{1}{| | A | |} \leq \frac{| | A | |}{| | b | |}$
$| | x | | \leq | | A^{- 1} | | \cdot | | b | |$
$\frac{1}{| | A^{- 1} | | \cdot | | b | |} \leq \frac{1}{| | x | |}$
$| | Δ x | | \leq | | A^{- 1} | | \cdot | | Δ b | |$
$| | Δ b | | \leq | | A | | \cdot | | Δ x | |$

To get the first inequality, multiply $2$ and $4$ . To get the second, multiply $1$ and $3$ .

Theorem

Let $| | \cdot | |$ be a matrix norm on $M_{n}$ . Suppose $A, Δ A \in M_{n}$ with $A$ invertible and $| | A^{- 1} | | \cdot | | Δ A | | < 1$ . Then $A + Δ A$ is invertible. Define $Δ (A^{- 1}) := A^{- 1} - (A + Δ A)^{- 1}$ . And

\frac{| | Δ (A^{- 1}) | |}{| | A^{- 1} | |} \leq \frac{κ (A) \frac{| | Δ A | |}{| | A | |}}{1 - κ (A) \frac{| | Δ A | |}{| | A | |}}

Proof

We suppose that
$| | - A^{- 1} Δ A | | \leq | - 1 | \cdot | | A^{- 1} | | \cdot | | Δ A | | < 1$ . So by the previous theorem matrices with norm less than 1 define an infinite series inverse, we have that $I - (- A^{- 1} Δ A)$ is invertible and $(I - (- A^{- 1} Δ A))^{- 1} = \sum_{i = 1}^{\infty} (- A^{- 1} Δ A)^{i}$ .

Thus, $A + Δ A = A (I + A^{- 1} Δ A)$ is invertible! and $(A + Δ A)^{- 1} = [\sum_{i = 1}^{\infty} (- A^{- 1} Δ A)^{i} A^{- 1}]$

$Δ (A^{- 1}) = A^{- 1} - (A + Δ A)^{- 1} = - \sum_{i = 1}^{\infty} (- A^{- 1} Δ A)^{i} A^{- 1}$ so we have
$| | Δ (A^{- 1}) | | \leq | | - \sum_{i = 1}^{\infty} (- A^{- 1} Δ A)^{i} A^{- 1} | | \leq \sum_{i = 1}^{\infty} | | A^{- 1} Δ A | |^{i} | | A^{- 1} | |$ ie

\frac{\lvert \lvert \Delta(A^{-1}) \rvert \rvert }{\lvert \lvert A^{-1} \rvert \rvert } \leq \sum_{i=1}^\infty \lvert \lvert A^{-1}\Delta A \rvert \rvert {#i} = \frac{\lvert \lvert A^{-1}\Delta A \rvert \rvert }{1-\lvert \lvert A^{-1}\Delta A \rvert \rvert } \leq \frac{\lvert \lvert A^{-1} \rvert \rvert \cdot \lvert \lvert \Delta A \rvert \rvert \frac{\lvert \lvert A \rvert \rvert }{\lvert \lvert A \rvert \rvert }}{1- \lvert \lvert A^{-1} \rvert \rvert \cdot \lvert \lvert \Delta A \rvert \rvert \frac{\lvert \lvert A \rvert \rvert }{\lvert \lvert A \rvert \rvert } }

And the result follows immediately!

(see condition number determines the amount of noise in an inverse)