Lecture 07

[[lecture-data]]

2024-09-11

Readings

Recall: complex numbers.

Complex Conjugate

Let $k = α + β i$ be a complex number. Then the complex conjugate is $\bar{k} = α - β i$

(see complex conjugate)

Note that $\overset{―}{k_{1} + k_{2}} = \overset{―}{k_{1}} + \overset{―}{k_{2}}$ in this case

Now, suppose we have a complex number in polar coordinates $z = ρ e^{i θ}$ . Then the complex conjugate is $\overset{―}{z} = ρ e^{- i θ}$

Note that $\overset{―}{z_{1} z_{2}} = \overset{―}{z_{1}} \cdot \overset{―}{z_{2}}$ in this case

2. Chapter 2

Conjugate Transpose, Hermitian

Let $A \in M_{m \times n} (C)$ . Define the conjugate transpose of $A$ :

A^{*} = \overset{―}{A^{T}}

If $A^{*} = A$ , then we call $A$ hermitian

(see conjugate transpose, hermitian)

Note

If $A \in M_{m, n}$ and $B \in M_{n, p}$ then $(A B)^{T} = B^{T} A^{T}$ and $(A B)^{*} = B^{*} A^{*}$

Length

The euclidean length of a vector $z$ is

| | z | |_{(2)} = \sqrt{z^{*} z}

(see euclidean length of a vector)

Orthogonal, Orthonormal Vectors

If $x, y \in C$ , then $x ⊥ y$ or " $x$ is orthogonal to $y$ " means $y^{*} x = 0$ .

${x_{1}, \dots, x_{n}} \subset C^{n}$ "are orthogonal" means the $x_{i}$ are pairwise orthogonal. We say they are orthonormal if for all $i$ , we have that $x_{i}^{*} x_{i} = 1$

(see orthogonal vectors)

Unitary

A matrix $U \in M_{n}$ is called unitary precisely when $U^{*} U = I$ .

ie, $U$ is invertible
And its inverse is its conjugate transpose

If $Q \in M_{n} (R)$ , then $Q$ is (real) orthogonal precisely when $Q^{T} Q = I$

(see unitary matrices)

Note

A matrix $U \in M_{n}$ is unitary if and only if the columns of $U$ are orthonormal.

Proof

$U$ unitary means that $U^{*} U = I$ if and only if for all columns $u_{k}$ of $U$ :

u_{i}^{*} u_{j} = {\begin{cases} 1 if i = j \\ 0 otherwise \end{cases}

which means that the columns are 1) orthogonal and 2) normal

(see unitary matrices have orthonormal columns)

Note

If $U, V \in M_{n}$ are unitary,then $U V$ is unitary.

Proof

$(U V)^{*} (U V) = V^{*} U^{*} U V^{*} = I$

(see the product of unitary matrices is unitary)

Fact

Unitary matrices form a group

Note

If $U \in M_{n}$ is unitary, then $U : C^{n} \to C^{n}$ is an isometry. This means
$\forall x \in C^{n}$ , the length

| | U x | |_{2} = \sqrt{(U x)^{*} U x} = \sqrt{x^{*} U^{*} U x} = | | x | |_{2} $ $ T h a t i s, t h e t r a n s f o r m a t i o n p r e s e r v e s t h e l e n g t h o f a n y i n p u t v e c t o r . A d d i t i o n a l l y, f o r a n y p a i r o f v e c t o r s $ x, y \in C^{n} $ w e h a v e $ $ | | U x - U y | |_{2} = | | U (x - y) | |_{2} = | | x - y | |_{2}

by the result immediately above, so the transformation also preserves pairwise distances between input vectors!

(see unitary matrices define isometries)

Unitarily Similar

$A, B \in M_{n}$ are unitarily similar (or unitarily equivalent)means that there exists $U \in M_{n}$ unitary such that $A = U B U^{*}$

This is a similarity where the change of basis is a distance-preserving one.

(see unitarily similar)

Frobenius Norm

"treat the matrix like a long euclidean vector"

| | A | |_{F} = \sqrt{\sum_{i, j} | a_{i j} |^{2}}

(see frobenius norm)

Note

Suppose $A, B$ are unitarily similar. Then $| | A | |_{F} = | | B | |_{F}$

Proof

We have that $A = U B U^{*}$ for $U$ unitary

\begin{aligned} | | A | |_{F}^{2} & = Tr (A^{*} A) \\ = Tr ((U B U^{*})^{*} U B U^{*}) \\ = Tr (U B^{*} U^{*} U B U^{*}) \\ = Tr (U B^{*} B U^{*}) \\ = Tr (U^{*} U B^{*} B) \\ = Tr (B^{*} B) & = | | B | |_{F}^{2} \end{aligned}

(see unitarily similar matrices have the same frobenius norm)

Householder Transformation

Suppose we have $w \in C^{n}$ with $| | w | | = 1$ . The Householder transformation is given by

H_{w} := I - 2 w w^{*}

(see householder transformation)

$H_{w}$ is hermitian and unitary!

\begin{aligned} (I - 2 w w^{*}) (I - 2 w w^{*}) & = I - 4 w w^{*} + 4 w w^{*} w w^{*} \\ = I - 4 w w^{*} + 4 w w^{*} \\ = I \end{aligned}

Note

Given any unit vector $x \in C^{n}$ , there exists a unitary matrix $U \in M_{n}$ where the first column is $x$ .

Proof

Gram-Schmidt: start with the first column $x$ and any orthonormal basis for the rest of the matrix.

(see we can find a unitary matrix for any unit vector)

Note

Let $x, y \in R^{n}$ such that $x \neq y$ but $| | x | | = | | y | |$ . If we take $w = \frac{x - y}{| | x - y | |}$ , then $H_{w} x = y$ and $H_{w} y = x$

if $x$ is real, then $$Hx = \begin{bmatrix}
1 \
0 \
\vdots \
0
\end{bmatrix}$$ use the unitary matrix $U = I$ . Otherwise, take $U = H_{w}$ such that $H_{w} [1 0 0 \dots 0]^{T} = x$ and use $$w = \frac{1}{\lvert \lvert \begin{bmatrix}
1 \
0 \
\vdots \
0
\end{bmatrix} - x \rvert \rvert_{2} } \left(\begin{bmatrix}
1 \
0 \
\vdots \
0
\end{bmatrix} - x\right)$$

central result for chapter 2: Schur's Theorem (next lecture)