Definition
a function from $X\times X \to \mathbb{C}$ (denoted by $\langle \cdot, \cdot \rangle$) satisfies the following properties:
- $\langle x, x \rangle \geq 0$ and $\langle x, x \rangle = 0 \Leftrightarrow x = 0$
- $(\forall x, y \in X)$ $\langle x, y \rangle = \overline{\langle y, x \rangle}$
- $(\forall \alpha, \beta \in \mathbb{C})$ $(x, y, z \in X), \langle z,\alpha x + \beta y, \rangle = \alpha \langle z, x \rangle + \beta \langle z, y \rangle$
Remark:
- (2) says the inner product is linear in the second variable
- (3) says the inner product is sesquilinear
- (2) and (3) implies $\langle \alpha x + \beta y, z \rangle = \overline{\alpha} \langle x, z \rangle + \overline{\beta} \langle y, z \rangle$, so the inner product is conjugate linear in the first variable
Definition
An inner product space which is complete with respect to the norm induced by the inner product is called a Hilbert space.
Examples
Example of Hilbert space
- any finite dimensional inner product space
- $L^2={(x1, x_2, \cdots) | x_k \in \mathbb{C}, \sum{k=1}^{\infty} |xk|^2 < \infty}$ with $\langle y, x \rangle = \sum{k=1}^{\infty} \overline{y_k}x_k$
- $L^2(A)$ for any measurable $A \subset \mathbb{R}^n$, with $\langle g, f \rangle = \int_A \overline{g(x)}f(x)dx$
Example of incomplete inner product space
- $C([a, b])$ with $\langle g, f \rangle = \int_a^b \overline{g(x)}f(x)dx$
- $C([a, b])$ with this inner product is not complete; it is dense in $L^2([a, b])$, which is complete
Useful Laws and Inequalities
Parallelogram Law
Let $X$ be an inner product space. Then $(\forall x, y \in X)$
Polarization Identity
Let $X$ be an inner product space. Then $(\forall x, y \in X)$
Note: In a real inner product space, $\langle x, y \rangle = \frac{1}{4}(|x + y|^2 - |x - y|^2)$
Theorem
Suppose $(X, ||\cdot||)$ is a normed linear space.
The norm $||\cdot||$ is induced by an inner product $\Leftrightarrow$ the parallelogram law holds in $(X, ||\cdot||)$.
The Pythagorean Theorem
If $x_1, x_2, \cdots, x_n\in X$ and $x_j \perp x_k$ for $j\neq k$, then
Convex Sets
A subset $A$ of a vector space $X$ is called convex if $(\forall x, y \in A)$ and $(\forall t \in (0, 1))$, $(1-t)x + ty \in A$.
Theorem
Every nonempty closed convex subset $ of a Hilbert space $ has a unique element of smallest norm.
Corollary
If $A$ is a nonempty closed convex set in a Hilbert space and $x\in A$, then $\exists$ a unique closest element of $A$ to $x$.
Orthogonal Sets
Let $X$ be an inner product space. Let $A$ be a set (not necessarily countable).
A set ${u\alpha}{\alpha \in A}\subset X$ is called orthogonal set if $(\forall \alpha\neq \beta)$ $\langle u\beta, u\alpha \rangle = 0$. (In particular, $u_\alpha \neq 0$)
Theorem
If ${u1, u_2, \cdots, u_k}$ is an orthogonal set in an inner product space $X$, and $x=\sum{j=1}^k cju_j$, then $c_j = \langle u_j, x \rangle$ for $1\leq j\leq k$ and $||x||^2 = \sum{j=1}^l |c_j|^2$.
Theorem. (Gram-Schmidt process)
Let $V$ be a subspace of an inner product space $X$, and suppose $V$ has a finite or countable basis ${xn}{n\geq 1}$.
Then $V$ has a basis ${un}{n\geq 1}$ which is orthogonal. moreover, we can choose the ${un}{n\geq 1}$ so that $\forall m\geq 1$, $\text{span}{u_1, u_2, \cdots, u_m} = \text{span}{x_1, x_2, \cdots, x_m}$.
Theorem
Let $V$ be a finite dimensional subspace of a Hilbert space $X$.
Let ${u1, u_2, \cdots, u_n}$ be a basis for $V$ which is orthogonal, and let $P$ be the orthogonal projection of $X$ onto $V$.
Then $Px = \sum{j=1}^n \langle uj, x \rangle u_j$ and $||x||^2 = ||Px||^2 + ||Qx||^2= \sum{j=1}^n |\langle u_j, x \rangle|^2 + ||Qx||^2$.
Bessel’s Inequality
Let ${u\alpha}{\alpha\in A}$ be an orthogonal set in a Hilbert space $X$, let $x\in X$, and let $\hat{x(\alpha)} = \langle u\alpha, x \rangle$.
Then $\sum{\alpha\in A} |\hat{x(\alpha)}|^2 \leq ||x||^2$.