1 Inner Product Space - WordPress.com · All inner product spaces are normed spaces for the norm de...

12
MAT641- MSC Mathematics, MNIT Jaipur Functional Analysis-Inner Product Space and orthogonality Dr. Ritu Agarwal Department of Mathematics, Malaviya National Institute of Technology Jaipur April 11, 2017 1 Inner Product Space Inner product on linear space X over field K is defined as a function h.i : X × X -→ K which satisfies following axioms: For x, y, z X , α K , i. hx + y,z i = hx, z i + hy,z i; ii. hαx, y i = αhx, y i; iii. hx, y i = hy,xi; iv. hx, xi≥ 0, hx, xi = 0 iff x = 0. An inner product defines norm as: kxk = p hx, xi and the induced metric on X is given by d(x, y )= kx - y k = p hx - y,x - y i Further, following results can be derived from these axioms: hαx + βy,z i = αhx, z i + β hy,z i hx, αy i = αhx, y i hx, αy + βz i = αhx, y i + β hx, z i α, β are scalars and bar denotes complex conjugation. Remark 1.1. Inner product is linear w.r.to first factor and conjugate linear (semilin- ear) w.r.to second factor. Together, this property is called ‘Sesquilinear’ (means one and half-linear). Date: April 11, 2017 Email:[email protected] Web: drrituagarwal.wordpress.com

Transcript of 1 Inner Product Space - WordPress.com · All inner product spaces are normed spaces for the norm de...

Page 1: 1 Inner Product Space - WordPress.com · All inner product spaces are normed spaces for the norm de ned by kxk= p hx;xi. Also, show that the triangle inequality for norm can be obtained

MAT641- MSC Mathematics, MNIT Jaipur

Functional Analysis-Inner Product Space and orthogonality

Dr. Ritu AgarwalDepartment of Mathematics,

Malaviya National Institute of Technology Jaipur

April 11, 2017

1 Inner Product Space

Inner product on linear space X over field K is defined as a function 〈.〉 : X ×X −→ Kwhich satisfies following axioms:For x, y, z ∈ X, α ∈ K,

i. 〈x+ y, z〉 = 〈x, z〉+ 〈y, z〉;

ii. 〈αx, y〉 = α〈x, y〉;

iii. 〈x, y〉 = 〈y, x〉;

iv. 〈x, x〉 ≥ 0, 〈x, x〉 = 0 iff x = 0.

An inner product defines norm as:

‖x‖ =√〈x, x〉

and the induced metric on X is given by

d(x, y) = ‖x− y‖ =√〈x− y, x− y〉

Further, following results can be derived from these axioms:

〈αx+ βy, z〉 = α〈x, z〉+ β〈y, z〉

〈x, αy〉 = α〈x, y〉〈x, αy + βz〉 = α〈x, y〉+ β〈x, z〉

α, β are scalars and bar denotes complex conjugation.

Remark 1.1. Inner product is linear w.r.to first factor and conjugate linear (semilin-ear) w.r.to second factor. Together, this property is called ‘Sesquilinear’ (means oneand half-linear).

Date: April 11, 2017

Email:[email protected] Web: drrituagarwal.wordpress.com

Page 2: 1 Inner Product Space - WordPress.com · All inner product spaces are normed spaces for the norm de ned by kxk= p hx;xi. Also, show that the triangle inequality for norm can be obtained

MAT641-Inner Product Space and orthogonality 2

Definition 1.2 (Hilbert space). A complete inner product space is known as HilbertSpace.

Example 1. Examples of Inner product spaces

i. Euclidean Space Rn, 〈x, y〉 = ξ1η1 + ...+ ξnηn, x = (ξ1, ..., ξn), y = (η1, ..., ηn) ∈ Rn.

ii. Unitary Space Cn, 〈x, y〉 = ξ1η1 + ...+ ξnηn, x = (ξ1, ..., ξn), y = (η1, ..., ηn) ∈ Cn.

iii. Function space L2[a, b], 〈x, y〉 =∫ ba x(t)y(t)dt, x(t) and y(t) are complex valued

functions (May be real valued also).

iv. l2 = {x = (ξ1, ξ2, ...) :∑∞

i=1 |ξi|2 <∞} is an IPS for 〈x, y〉 =∑∞

i=1 ξiηi.

2 Norm and Parallelogram equality

To prove triangle inequality for normed space, first we are required to prove the Schwarzinequality.

Theorem 2.1 (Schwarz inequality).

|〈x, y〉| ≤ ‖x‖‖y‖ (2.1)

where, equality holds if and only if {x, y} is linearly dependent set.

Proof. If y = 0, then (2.1) holds since 〈x, 0〉 = 0. Let y 6= 0. For every scalar α, wehave

0 ≤ ‖x− αy‖2 = 〈x− αy, x− αy〉= 〈x, x〉 − α〈x, y〉 − α[〈y, x〉 − α〈y, y〉]

(2.2)

Choosing α = 〈y,x〉〈y,y〉 so that the expression in brackets vanish. We get

0 ≤ 〈x, x〉 − 〈y, x〉〈y, y〉

〈x, y〉 = ‖x‖2 − |〈x, y〉|2

‖y‖2(Using〈y, x〉 = 〈x, y〉) (2.3)

Rearranging the terms, we get (2.1).Equality holds if ‖x− αy‖ = 0 i.e. x = αy, {x, y} are linearly dependent.

Theorem 2.2. All inner product spaces are normed spaces for the norm defined by‖x‖ =

√〈x, x〉. Also, show that the triangle inequality for norm can be obtained from

inner product as‖x+ y‖ ≤ ‖x‖+ ‖y‖ (2.4)

where, equality holds if and only if {x, y} is linearly dependent set x = cy, c ≥ 0 ory = 0.

Page 3: 1 Inner Product Space - WordPress.com · All inner product spaces are normed spaces for the norm de ned by kxk= p hx;xi. Also, show that the triangle inequality for norm can be obtained

MAT641-Inner Product Space and orthogonality 3

Proof. (i) From axiom (iv) for inner product space, we have ‖x‖ =√〈x, x〉 ≥ 0 and

‖x‖ =√〈x, x〉 = 0 iff x = 0.

(ii) ‖αx‖ =√〈αx, αx〉 =

√αα〈x, x〉 =

√|α|2〈x, x〉 = |α| ‖x‖.

(iii) Triangle inequality

‖x+ y‖2 = 〈x+ y, x+ y〉 = 〈x, x〉 + 〈x, y〉 + 〈y, x〉 + 〈y, y〉= ‖x‖2 + 〈x, y〉 + 〈y, x〉 + ‖y‖2

(2.5)

Using Schwarz inequality (2.1)

‖x+ y‖2 ≤ ‖x‖2 + 2|〈x, y〉|+ ‖y‖2 ≤ ‖x‖2 + 2‖x‖ ‖y‖+ ‖y‖2 = (‖x‖+ ‖y‖)2(2.6)

Taking square root on both sides, we get(2.4).

Further, equality holds in this derivation if and only if

〈x, y〉+ 〈y, x〉 = 2‖x‖ ‖y‖

i.e. 〈x, y〉+ 〈y, x〉 = 2<〈x, y〉 = 2‖x‖ ‖y‖. But Schwarz inequality says that‖x‖ ‖y‖ ≥ |〈x, y〉|, so that <〈x, y〉 ≥ |〈x, y〉|.

Since the real part of a complex number cannot exceed the absolute value, we musthave equality, which implies linear dependence from (2.1), say y = 0 or x = cy.With the equality sign, we have <〈x, y〉 = |〈x, y〉|, and hence imaginary part must bezero.

If x = cy, <(〈x, y〉) = <(〈cy, y〉) = <(c〈y, y〉) ≥ 0 ⇒ c > 0.

Remark 2.3. Every Hilbert space is Banach space. However converse is not necessarilytrue.

2.1 Parallelogram equality

A norm on an inner product space satisfies the important Parallelogram equality

‖x+ y‖2 + ‖x− y‖2 = 2(‖x‖2 + ‖y‖2) (2.7)

Proof.

LHS = ‖x+ y‖2 + ‖x− y‖2

= 〈x+ y, x+ y〉+ 〈x− y, x− y〉= 〈x, x〉+ 〈x, y〉+ 〈y, x〉+ 〈y, y〉+ 〈x, x〉 − 〈x, y〉+ 〈y, y〉 − 〈y, x〉= 2(〈x, x〉+ 〈y, y〉)= 2(‖x‖2 + ‖y‖2)

(2.8)

Page 4: 1 Inner Product Space - WordPress.com · All inner product spaces are normed spaces for the norm de ned by kxk= p hx;xi. Also, show that the triangle inequality for norm can be obtained

MAT641-Inner Product Space and orthogonality 4

Examples of normed spaces which are not inner product spaces:

Example 2. Sequence space lp, p 6= 2Consider x = (1, 1, 0, 0, 0, ....) and y = (1,−1, 0, 0, 0, ...) ∈ lp and calculate

‖x‖ = ‖y‖ = 21/p, ‖x+ y‖ = ‖x− y‖ = 2

Parallelogram equality is not satisfied. Hence lp, p 6= 2 is a Banach space but notHilbert space.

Example 3. Space C[a,b] is not an inner product space for ‖x‖ = maxt∈J |x(t)|, J =[a, b].Consider x(t) = 1 and y(t) = (t − a)/(b − a). We have, ‖x‖ = 1, ‖y‖ = 1 and‖x(t) + y(t)‖ = ‖1 + t−a

b−a‖ = 2, ‖x(t)− y(t)‖ = ‖1− t−ab−a‖ = 1.

Thus, Parallelogram equality is not satisfied.

Theorem 2.4 (Polarization identity). The inner product in term of induced norm isgiven by

1. For a real IPS X,

〈x, y〉 =1

4(‖x+ y‖2 − ‖x− y‖2) (2.9)

2. For a complex IPS

<〈x, y〉 =1

4(‖x+ y‖2 − ‖x− y‖2) (2.10)

and

=〈x, y〉 =1

4(‖x+ iy‖2 − ‖x− iy‖2) (2.11)

Proof is left as an exercise to the reader.

3 Continuity in IPS

Theorem 3.1 (Continuity). If in an inner product space xn → x and yn → y, then〈xn, yn〉 → 〈x, y〉.

Proof. Consider

|〈xn, yn〉 − 〈x, y〉| = |〈xn, yn〉 − 〈xn, y〉+ 〈xn, y〉 − 〈x, y〉| = |〈xn, yn − y〉|+ |〈xn − x, y〉|

Applying Schwarz inequality

|〈xn, yn〉 − 〈x, y〉| ≤ ‖xn‖ ‖yn − y‖+ ‖xn − x‖ ‖y‖ → 0 as n→∞

Hence inner product is a continuous function.

Page 5: 1 Inner Product Space - WordPress.com · All inner product spaces are normed spaces for the norm de ned by kxk= p hx;xi. Also, show that the triangle inequality for norm can be obtained

MAT641-Inner Product Space and orthogonality 5

Definition 3.2 (Isomorphism). An isomorphism T of an inner product space (X, 〈.〉)onto an IPS (X, 〈.〉0) over the same field K of scalars is a bijective linear operatorT : X → X which preserves the values of the inner products i.e. 〈Tx, Ty〉0 = 〈x, y〉.

Exercise 3.3. Let Y be a subspace of a Hilbert space H.

1. Y is complete iff Y is closed in H.

2. If Y is a finite dimensional subspace then Y is complete.

3. Every subspace of a separable space is separable.

Theorem 3.4 (Minimizing Vector). Let X be an inner product space and Y 6= φ is aconvex subset which is complete (in the metric induced by the inner product) subset.Then for every given x ∈ X, there is an unique y ∈ Y such that

δ = infy′∈Y‖x− y′‖ = ‖x− y‖.

Proof: (Existence of y): Given δ = infy′∈Y ‖x− y′‖. By definition of infimum, thereexists a sequence (yn) in Y such that

δn = ‖x− yn‖ δn → δ as n→∞

We will show that (yn) is a Cauchy sequence in Y.Let x− yn = vn. Then

‖vn + vm‖ = ‖2x− (yn + ym)‖ = 2

∥∥∥∥x− (yn + ym)

2

∥∥∥∥But yn, ym ∈ Y , (yn + ym)/2 ∈ Y as Y is convex. Hence

‖vn + vm‖ ≥ 2δ. (3.1)

Now ‖yn− ym‖2 = ‖vn− vm‖2 = 2‖vn‖2 + 2‖vm‖2−‖vn− vm‖2 (by Parallelogram law).⇒ ‖yn− ym‖2 = 2‖x− yn‖2 + 2‖x− ym‖2−‖vn− vm‖2 ≤ 2δ2n + 2δ2m− 4δ2 (using (3.1)).As n,m→∞, δn → δ and δm → δ. Thus‖yn − ym‖ → 0 as n,m → ∞ i.e. (yn) is a Cauchy sequence. Since Y is complete,(yn)→ y ∈ Y . Thus existence of y is guaranteed.

(Uniqueness): Let y0 and y1 ∈ Y such that

δ = ‖x− y0‖ = ‖x− y1‖

. Now ‖y1− y0‖ = ‖(y1− x)− (y0− x)‖ ≤ 2‖y1− x‖2 + 2‖y0− x‖2−‖y1− x+ y0− x‖2(by Parallelogram law)⇒ ‖y1 − y0‖ ≤ 2δ2 + 2δ2 − 4δ2 → 0.But ‖y1 − y0‖ ≥ 0, we have ‖y1 − y0‖ = 0 i.e. y1 = y0.

Page 6: 1 Inner Product Space - WordPress.com · All inner product spaces are normed spaces for the norm de ned by kxk= p hx;xi. Also, show that the triangle inequality for norm can be obtained

MAT641-Inner Product Space and orthogonality 6

4 Orthogonality

Definition 4.1 (Orthogonality). An element x of an inner product space X is said tobe orthogonal to an element y ∈ X if

〈x, y〉 = 0

and we write x⊥y.

Theorem 4.2 (Orthogonality). Let X be an inner product space and Y 6= φ is a convexcomplete (in the metric induced by the inner product) subset. Then for a fixed x ∈ X,z = x− y is orthogonal to Y where

‖x− y‖ = δ = infy′∈Y‖x− y′‖, y ∈ Y.

Proof: Let z ⊥ Y is false. Then there exist a y′ ∈ Y such that

〈z, y′〉 = β 6= 0

Clearly, y′ 6= 0 since otherwise 〈z, y′〉 = 0.Consider the norm

‖z − αy′‖2 = 〈z − αy′, z − αy′〉 = 〈z, z〉 − α〈z, y′〉 − α(〈z, y′〉 − α〈y′, y′〉)

Choose α = 〈z,y′〉〈y′,y′〉 = β

‖y′‖2 so that

‖z − αy′‖2 = ‖z‖2 − ββ

‖y′‖2< ‖z‖2 = δ2

We claim, that ‖z − αy′‖2〈δ2 can never happen. Since

‖z − αy′‖ = ‖x− y − αy′‖ = ‖x− αy1‖ ≥ δ

y1 = y+αy′ and δ = infy∈Y ‖x−y‖. This shows contradiction. Hence β = 0⇒ 〈z, y〉 = 0i.e. (x− y) ⊥ Y .

4.1 Orthogonal Compliments and Direct sums

Definition 4.3 (Direct Sum). A vector space X is said to be direct sum of two sub-spaces Y and Z of X, written X = Y ⊕ Z, if each x ∈ X has a unique representationx = y + z.Z is called algebraic complement of Y in X and vice versa and Y, Z is called a comple-mentary pair of subspaces in X.

Definition 4.4 (Anihilator). An orthogonal compliment of Y 6= φ in an IPS X is theset Y ⊥ = {x ∈ X : x ⊥ Y }, called anihilator.

Page 7: 1 Inner Product Space - WordPress.com · All inner product spaces are normed spaces for the norm de ned by kxk= p hx;xi. Also, show that the triangle inequality for norm can be obtained

MAT641-Inner Product Space and orthogonality 7

Theorem 4.5. (i) Y ⊥ is a vector subspace.(ii) Y ⊥ is a closed subspace.(iii) Y ⊂ Y ⊥⊥.

Main concern is for a Hilbert space H, a closed subspace Y and its ’OrthogonalCompliment’

Y ⊥ = {z ∈ H|z ⊥ Y }

Theorem 4.6 (Projection theorem). Let Y be any closed convex subspace of a Hilbertspace H. Then H = Y ⊕ Y ⊥.

Proof: Since H is complete and Y is closed, Y is complete. By Theorem 4.2, forevery x ∈ H, there is a y ∈ Y such that

x = y + z z ∈ Z = Y ⊥

To prove uniqueness, assume that x = y + z = y′ + z′, where y, y′ ∈ Y and z, z′ ∈ Z.Since y−y′ ∈ Y and z− z′ ∈ Z, we see that y−y′ ∈ Y ∩Y ⊥ = {0}. This implies y = y′

and z = z′. Hence proved.

Definition 4.7 (Orthogonal Projection). Define a mapping P : H → Y i.e. Px = y. Pis called ‘Orthogonal Projection’ of H onto Y. Obviously P is a bounded linear operator.It maps H onto Y, Y onto itself and Z = Y ⊥ onto {0}.P is idempotent, i.e. P 2 = P .

Lemma 4.8 (Null space). The orthogonal compliment Y ⊥ of a closed subspace Y of aHilbert space H is the null space N(P ) of the orthogonal projection P of H onto Y .

Proof. Since Y is a closed subspace of H, H = Y ⊕ Y ⊥. Therefore any x ∈ H can bewritten as x = y+z where y ∈ Y and z ∈ Y ⊥. If P : H → Y is the projection mapping,Px = y, x ∈ H. In particular let x ∈ Y ⊥ then x = 0 + z = z since 〈x, y〉 = 0. HencePz = 0. z is arbitrary element of Y ⊥. Hence the theorem.

Lemma 4.9 (Closed subspace). If Y is a closed subspace of a Hilbert space H, thenY = Y ⊥⊥

Proof. To prove Y = Y ⊥⊥, we need to show that Y ⊂ Y ⊥⊥ and Y ⊃ Y ⊥⊥.Let x ∈ Y ⇒ x ⊥ Y ⊥ ⇒ x ∈ Y ⊥⊥. That is Y ⊂ Y ⊥⊥.

To show Y ⊃ Y ⊥⊥, let x ∈ Y ⊥⊥. Since Y ⊥⊥ is a vector space, y ∈ Y ⊂ Y ⊥⊥,z = x− y ∈ Y ⊥⊥. Hence z ⊥ Y ⊥ but z ∈ Y ⊥.Hence z = 0 so that x = y i.e. x ∈ Y .Since x is arbitrary, Y ⊃ Y ⊥⊥.

Lemma 4.10 (Dense Set (3.3-7)). For any subset M 6= φ of a Hilbert space H, the spanof M is dense in H if and only if M⊥ = {0}.

Page 8: 1 Inner Product Space - WordPress.com · All inner product spaces are normed spaces for the norm de ned by kxk= p hx;xi. Also, show that the triangle inequality for norm can be obtained

MAT641-Inner Product Space and orthogonality 8

Proof. Let x ∈M⊥ and assume V = spanM to be dense in H. Then x ∈ V = H. Thus,there exists a sequence (xn) in V such that xn → x. Since x ∈ M⊥ and M⊥ ⊥ V , wehave 〈xn, x〉 = 0. The continuity of the inner product implies that 〈xn, x〉 → 〈x, x〉 =‖x‖2 = 0, so that x = 0. Since x ∈M⊥ is arbitrary, this shows that M⊥ = {0}.

Conversely, suppose that M⊥ = {0}. If x ⊥ V=Span M, then x ⊥M so that x ∈M⊥

and x = 0. Hence V ⊥ = {0}. H being Hilbert space, can be written as H = Y ⊕Y ⊥, Yis closed. Noting that V is subspace of H, taking Y = V , we thus obtain H = V .

5 Orthonormal sets and sequences

An orthogonal set M in an inner product space X is a subset M ⊂ X whose elementsare pairwise orthogonal.

An orthonormal set M in an inner product space X is a subset M ⊂ X is an orthog-onal set in X whose elements have norm 1. That is for all x, y ∈M

〈x, y〉 =

{0 x 6= y

1 x = y(5.1)

If an orthogonal or orthonormal set M is countable, we can arrange it in a sequence(xn) and call it orthogonal or orthonormal sequence.

For orthogonal elements x, y, we have 〈x, y〉 = 0 so that parallelogram equality leadsto

‖x+ y‖2 = ‖x‖2 + ‖y‖2 (5.2)

which is Pythogorean relation.More generally, if {x1, x2, ...xn} is an orthogonal set then

‖x1 + ...+ xn‖2 = ‖x1‖2 + ...+ ‖xn‖2 (5.3)

Lemma 5.1 (Linear independence). An orthonormal set is linearly independent.

Proof. Let {e1, e2, ..., en} be orthonormal. Consider the equation

α1e1 + α2e2 + ...+ αnen = 0

Taking inner product with ej, 1 ≤ j ≤ n, we get

〈n∑k=1

αkek, ej〉 =n∑k=1

αk〈ek, ej〉 = αj〈ej, ej〉 = αj = 0

This proves the linear independence for any finite orthonormal set.

Example 4. In the space R3 the basis (1,0,0), (0,1,0), (0,0,1) is an orthonormal basis.

Page 9: 1 Inner Product Space - WordPress.com · All inner product spaces are normed spaces for the norm de ned by kxk= p hx;xi. Also, show that the triangle inequality for norm can be obtained

MAT641-Inner Product Space and orthogonality 9

Example 5. In Example 1(iv), e1, .., en, ... is the orthonormal sequence in l2 where

en = (0, 0, ..., 1︸︷︷︸nth position

, 0, ...)

Example 6. Let X be the IPS of all real valued continuous functions on [0, 2π] withinner product defined by

〈x, y〉 =

∫ 2π

0

x(t) y(t)dt (5.4)

An orthogonal sequences in X are un = cosnt and vn = sinnt. That is 〈un, um〉 = 0 forn 6= m. Also 〈vn, vm〉 = 0 for n 6= m. If we write e0 = u0/

√2π and en = cosnt√

π, en

sinnt√π

,

we get an orthonormal sequence of functions {e0, en, en}.

5.1 Advantage of orthonormal sequence

Expressing elements of IP space X in terms of orthonormal basis actually makes thedetermination of coefficients very easy as compared to arbitrary linearly independentset. Let e1, .., en, ... is the orthonormal sequence in X. Then x ∈ X can be written as

x =n∑k=1

αkek

〈x, ej〉 =n∑k=1

αk〈ek, ej〉 = αj

With these coefficients, we can write

x =n∑k=1

〈x, ek〉ek

Theorem 5.2 (Bessel inequality). Let (ek) be an orthonormal sequence in an innerproduct space X. Then for every x ∈ X

∞∑k=1

|〈x, ek〉|2 ≤ ‖x‖2 (5.5)

The inner product 〈x, ek〉 in (5.5) are called Fourier Coefficients of x with respectto orthonormal sequence (ek).

Proof. Let y =∑n

k=1 |αkek where n is fixed. Here αk = 〈y, ek〉 as (ek) is orthonormalsequence. We claim that for a particular choice of αk = 〈x, ek〉, k = 1, 2, ..., n, we shallobtain a y ∈ Y such that z = x− y ⊥ Y . By orthonormality,

‖y‖2 = 〈n∑k=1

〈x, ek〉ek,n∑j=1

〈x, ej〉ej〉 =n∑k=1

|〈x, ek〉|2

Page 10: 1 Inner Product Space - WordPress.com · All inner product spaces are normed spaces for the norm de ned by kxk= p hx;xi. Also, show that the triangle inequality for norm can be obtained

MAT641-Inner Product Space and orthogonality 10

Using this, we can show that z ⊥ y ∈ Y as

〈z, y〉 = 〈x−y, y〉 = 〈x, y〉−〈y, y〉 =

⟨x,

n∑k=1

〈x, ek〉ek

⟩−‖y‖2 =

n∑k=1

〈x, ek〉〈x, ek〉−n∑k=1

|〈x, ek〉|2 = 0

Using the Pythagorean relation

‖x‖2 = ‖y‖2 + ‖z‖2

here, we get

‖z‖2 = ‖x‖2 − ‖y‖2 = ‖x‖2 −n∑k=1

|〈x, ek〉|2

Since ‖z‖2 ≥ 0, for n ∈ N, we have

‖x‖2 ≥n∑k=1

|〈x, ek〉|2

Gram-Schmidt orthogonalization process

6 Total orthonormal sets

Definition 6.1. A total set (or fundamental set) in a normed space X is a subsetM ⊂ X whose span is dense in X. Accordingly an orthonormal set (or sequence orfamily) in an IPS X which is total in X is called a total orthonormal set (or sequenceor family) in X.

Remark 6.2. M is total in X if and only if span M = X.

Remark 6.3. A total orthonormal family in X is sometimes called an orthonormal basisfor X but not in sense of algebra except for the finite dimensional spaces.

Lemma 6.4. In every Hilbert space H 6= {0}, there exists a total orthonormal set.

Proof: For a finite dimensional space, this is clear. For an infinite dimensionalseparable space H, it follows from Gram-Schmidt orthogonalization process. For anon-separable H, a non-constructive proof follows from Zorn’s lemma.

Definition 6.5 (Hilbert Dimension). All total orthonormal sets in a given Hilbertspace H 6= {0} have the same cardinality. The latter is called the ‘Hilbert dimension’or orthogonal dimension of H.

Theorem 6.6 (Totality). Let M be a subset of IPS X. Then

Page 11: 1 Inner Product Space - WordPress.com · All inner product spaces are normed spaces for the norm de ned by kxk= p hx;xi. Also, show that the triangle inequality for norm can be obtained

MAT641-Inner Product Space and orthogonality 11

(a) If M is total in X, then there does not exist a non-zero x ∈ X which is orthogonalto every element of M. Briefly

x ⊥M ⇒ x = 0 (6.1)

(b) If X is complete, (6.1) is sufficient for the totality of M in X.

Proof. (a) Let H be completion of X [1, Theorem 3.2-3]. Then X is regarded as a densesubspace of H. By assumption, M is total in X so that span M is dense in X and hencedense in H. Lemma 4.10, orthogonal compliment of M in H is {0}. Therefore, if x ∈ Xand x ⊥M then x = 0.

(b) If X is a Hilbert space and M satisfies the given condition M⊥ = {0}, by Lemma4.10, the span of M is dense in X i.e. M is total in X.

Remark 6.7. The completeness of X in (b) is essential. If X is not complete, there maynot exist an orthonormal set M ⊂ X such that M is total in X. An example was givenby J. Diximier (1953).

Theorem 6.8 (Parseval relation). An orthonormal set M in a Hilbert space H is totalin H if and only if for all x ∈ H, the following Parseval relation holds

‖x‖2 =∑k=1

|〈x, ek〉|2 (6.2)

Proof. If M is not total, there is a non-zero x ⊥M in H. Since x ⊥M , in (6.2), we have〈x, ek〉 = 0 for all k, so that the RHS in (6.2) is zero whereas ‖x‖2 6= 0. This showsthat (6.2) does not hold.

Conversely assume that M is total in H. Consider any x ∈ H and its non-zero Fouriercoefficients arranged in a sequence 〈x, e1〉, 〈x, e2〉, .... We now define y as

y =∑k

〈x, ek〉ek, (6.3)

k may be finite or infinite as convergence follows from [1, Theorem 3.5-2].Let us show that x − y ⊥ M . Using the orthonormality, for every ej in (6.3), we

have〈x− y, ej〉 = 〈x, ej〉 −

∑k

〈x, ek〉〈ek, ej〉 = 〈x, ej〉 − 〈x, ej〉 = 0.

For every v ∈M not in (6.3), we have 〈x, v〉 = 0 and 〈ek, v〉 = 0 so that

〈x− y, v〉 = 〈x, v〉 −∑k

〈x, ek〉〈ek, v〉 = 0− 0 = 0.

Hence x− y ⊥ M i.e. x− y ∈ M⊥. Since M is total in H, M⊥ = {0}. This impliesx− y = 0 or x = y. Hence

‖x‖2 =

⟨∑k

〈x, ek〉ek,∑k

〈x, ek〉ek

⟩=∑k

〈x, ek〉〈x, ek〉 (6.4)

This completes the proof.

Page 12: 1 Inner Product Space - WordPress.com · All inner product spaces are normed spaces for the norm de ned by kxk= p hx;xi. Also, show that the triangle inequality for norm can be obtained

MAT641-Inner Product Space and orthogonality 12

Theorem 6.9. Let H be a Hilbert space. Then

(a) If H is separable, every orthonormal set is countable.

(b) If H contains an orthonormal sequence which is total in H, then H is separable.

Proof. (a) Let H be separable

References

[1] Erwin Kreyszig, Introductory Functional Analysis With Applications, Wiley India Edition,1989.

[2] Walter Rudin, Functional Analysis, Tata Mc-Graw Hill, New Delhi.

[3] Balmohan Vishnu Limaye, Functional analysis, New Age International, 1996.

[4] Aldric Loughman Brown and A. Page, Elements of functional analysis, Van Nostrand-Reinhold, 1970.

[5] J.B. Conway, A Course in Functional Analysis, 2nd Edition, Springer, Berlin, 1990.

Contents

1 Inner Product Space 1

2 Norm and Parallelogram equality 22.1 Parallelogram equality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

3 Continuity in IPS 4

4 Orthogonality 64.1 Orthogonal Compliments and Direct sums . . . . . . . . . . . . . . . . . . . . . . . . . . 6

5 Orthonormal sets and sequences 85.1 Advantage of orthonormal sequence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

6 Total orthonormal sets 10