MTH 664 Lectures 6, 7, & 8

MTH 664 0

MTH 664

Lectures 6, 7, & 8

Yevgeniy Kovchegov

Oregon State University

MTH 664 1

Topics:

• Measure and Integral.

• Random variables.

• Expectation.

• Jensen’s inequality.

• Convergence theorems.

MTH 664 2

Measure and Integral.

Definition 1. A collection F of subsets of Ω is a σ-algebra if

1. Ω ∈ F

2. A ∈ F implies Ac ∈ F

3. If A1, A2, A3, . . . ∈ F, then∞⋃j=1

Aj ∈ F

The elements of F are said to be F-measurable.

Note: F is an algebra if the latter condition is substituted withn⋃

j=1

Aj ∈ F for all n <∞.

MTH 664 3



1. Ω ∈ F


3. If A1, A2, A3, . . . ∈ F, then∞⋃j=1

Aj ∈ F

Properties:

• If A1, A2, A3, . . . , An ∈ F, thenn⋃

j=1

Aj ∈ F. This is shown by

taking An+1 = An+2 = . . . = ∅.

• If A1, A2, A3, . . . ∈ F, then by the de Morgan’s law,

∞⋂j=1

Aj ∈ F =

(∞⋃j=1

Acj

)c∈ F

MTH 664 4



1. Ω ∈ F


3. If A1, A2, A3, · · · ∈ F, then∞⋃j=1

Aj ∈ F

The elements of F are said to be F-measurable.

Examples:

• Trivial σ-algebra F = ∅,Ω.

• Let Ω = [0,1) and F = ∅, [0,1), [0,1/2), [1/2,1).

• If Ω is a topological space (e.g. Rn), the σ-algebra B generatedby all open sets is called a Borel σ-algebra, and the elements arecalled the Borel sets.

MTH 664 5


Definition 4. A set function µ : F → [0,∞] is called a measureon a measurable space (Ω,F) if

1.

µ(∅) = 0

2. For any sequence of disjoint (mutually non-intersecting)subsets A1, A2, . . . in F,

µ

(∞⋃j=1

Aj

)=

∞∑j=1

µ(Aj)

If µ(Ω) = 1, than µ is a probability measure. In this case, µ willsatisfy all the axioms of probability, and a measurable subsetA ∈ F is an event.

MTH 664 6


Definition 5. Suppose G is a collection of subsets of Ω. A setfunction µ : G → R is countably additive on G if for any sequenceof disjoint (mutually non-intersecting) subsets A1, A2, . . . in G,

µ

(∞⋃j=1

Aj

)=

∞∑j=1

µ(Aj)

Famous examples.

• Dirac point-mass.

δx(A) =

1 if x ∈ A0 if x 6∈ A

• Lebesgue measure m(·) on Ω = Rd.

m

([a1, b1)× · · · × [ad, bd)

)=

d∏j=1

(bj − aj)

MTH 664 7


Consider a measure space (Ω,F , µ), where Ω is the space, F isa σ-algebra, and µ : F → [0,∞] is a measure.

Definition 6. A function f : Ω→ R is said to be measurable if

f−1(B) = ω ∈ Ω : f(ω) ∈ B ∈ F for every Borel set B ∈ B.

Notation: f ∈ F.

Example. Let Ω = [0,1) and F = ∅, [0,1), [0,1/2), [1/2,1).Then, f(x) = x is not F-measurable. For a, b ∈ R, function

f(x) =a if x ∈ [0,1/2)b if x ∈ [1/2,1)

is F-measurable.

Almost sure equivalence: measurable functions f, g : Ω → Rare equal almost everywhere (a.e.) if

µ

(x ∈ Ω : f(x) 6= g(x)

)= 0.

Notation: f(x) = g(x) a.e.

MTH 664 8

Measure and Integral.Consider a measure space (Ω,F , µ), where Ω is the space, F isa σ-algebra, and µ : F → [0,∞] is a measure.

Partition y-axis into subintervals of size ≤ δ, i.e., 0 < yi+1−yi ≤ δ.

Recall: for a measurable function f over a set A ∈ F,

f−1[yi, yi+1) =x ∈ A : f(x) ∈ [yi, yi+1)

.

Lebesgue integral ∫A

f(ω) dµ(ω)

of a measurable function f over a set A ∈ F is defined as thelimit of ∑

i

y∗i µ(f−1[yi, yi+1)

), y∗i ∈ [yi, yi+1],

as δ → 0+.

Example. For A ∈ F, 1A(ω) is measurable, and∫Ω

1A(ω) dµ(ω) = µ(A).

MTH 664 9


Lebesgue integral ∫A

f(ω) dµ(ω)

of a measurable function f over a set A ∈ F is defined as thelimit of ∑

i

y∗i µ(f−1[yi, yi+1)

), y∗i ∈ [yi, yi+1],

as δ → 0+.

Properties:

• If f(x) = g(x) a.e. then∫Ω

f(x) dµ(x) =∫Ω

g(x) dµ(x).

• If f : Ω→ [−∞,∞] on A ∈ F such that µ(A), then∫A

f(x) dµ(x) = 0.

MTH 664 10


Monotone Convergence Theorem. Consider a measure space(Ω,F , µ), and a sequence fn : Ω → [0,∞] of nonnegative mea-surable functions satisfying

0 ≤ fn(x) ≤ fn+1(x)

for all n ∈ N and all x ∈ Ω. Then,

limn→∞

∫Ω

fn(x) dµ(x) =

∫Ω

limn→∞

fn(x) dµ(x).

Fatou’s Lemma. Consider a measure space (Ω,F , µ), and asequence fn : Ω → [0,∞] of nonnegative measurable functions.Then, ∫

Ω

lim infn→∞

fn(x) dµ(x) ≤ lim infn→∞

∫Ω

fn(x) dµ(x).

MTH 664 11


For a given a measure space (Ω,F , µ), a measurable functiong : Ω→ R is integrable if∫

Ω

|g(x)| dµ(x) <∞.

Lesbegue’s Dominated Convergence Theorem. Consider ameasure space (Ω,F , µ), and a sequence fn : Ω→ R of measur-able functions satisfying

fn(x)→ f(x) pointwise on Ω

and dominated by an integrable function g : Ω→ [0,∞), i.e.,

|fn(x)| ≤ g(x) a.e. ∀n.

Then,

limn→∞

∫Ω

fn(x) dµ(x) =

∫Ω

f(x) dµ(x).

MTH 664 12


Consider a measure space (Ω,F , µ). For p ≥ 1, space

Lp(Ω,F , µ) =

f ∈ F :

∫Ω

|f(x)|p dµ(x) <∞

is called the Lp-space.

The Lp-space is equipped with the Lp-norm:

‖f‖p =

(∫Ω

|f(x)|p dµ(x)

)1/p

for f ∈ Lp(Ω,F , µ).

Note that if ‖f − g‖p = 0, then f(x) = g(x) a.e.

Convergence in Lp-norm: fnLp−→ f if lim

n→∞‖f − fn‖p = 0.

MTH 664 13

Probability measure.

Let P be a probability measure on (Ω,F).Then the followingproperties can be shown.

(1) Monotonicity: If A ⊆ B then P (B) ≥ P (A)

(2) Subadditivity: P

(∞⋃j=1

Aj

)≤∞∑j=1

P (Aj)

(3) Continuity from below: If A1 ⊆ A2 ⊆ . . . then

limj→∞

P (Aj) = P

(∞⋃j=1

Aj

)(4) Continuity from above: If A1 ⊇ A2 ⊇ . . . then

limj→∞

P (Aj) = P

(∞⋂j=1

Aj

)

MTH 664 14

Product spaces.

If (Ωj,Fj, Pj) (j = 1,2, . . . , d) are probability spaces with corre-sponding probability measures. Let

Ω = Ω1 × · · · ×Ωd = (ω1, . . . , ωd) : ωj ∈ Ωj

be the product (sample) space. Then A1 × · · · × Ad : Ai ∈ Figenerates a σ-algebra over Ω,

F = F1 × · · · × Fd

together with the probability (product) measure

P (A1 × · · · ×Ad) = P1(A1) · P2(A2) · · ·Pd(Ad)

Examples. • Roll a die and toss a coin.

• Roll two dice.

• Toss a coin d times.

MTH 664 15

Random variables.

A real valued function X : Ω→ R is said to be a random variabledefined on (Ω,F) if it is F-measurable function, i.e.

X−1(B) = ω ∈ Ω : X(ω) ∈ B ∈ F

for any Borel set B ⊂ R.

Denote: X(ω) ∈ F

Example.

• Indicator variable. Take E ∈ F. Let 1E(ω) =

1 ω ∈ E0 ω 6∈ E

MTH 664 16

Random variables.

Consider a probability measure space(

Ω,F , P)

.

If X is a random variable, then X induces a probability measureon (R,B),

µ(B) = P

(X−1(B)

)= P

(ω ∈ Ω : X(ω) ∈ B

)for any B ∈ B (Borel σ-algebra).

This probability measure µ is called the (probability) distributionof r.v. X.

While the (cumulative) distribution function is defined as

F (a) = P (X ≤ a) = P(ω ∈ Ω : X(ω) ∈ (−∞, a]

)= µ

((−∞, a]

)

MTH 664 17

Distribution function.

The (cumulative) distribution function is defined as

F (a) = P (X ≤ a) = µ

((−∞, a]

)Properties:

I. F is nondecreasing

II. lima→∞

F (a) = 1, lima→−∞

F (a) = 0

III. F is right continuous: F (a+) = limx↓a

F (x) = F (a)

IV. F (a−) = limx↑a

F (x) = P (X < a)

V. P (X = a) = F (a)− F (a−)

Note: properties I-III suffice for a function F to be a distributionfunction for some r.v.

MTH 664 18

Continuous random variables.

X is a continuous random variable if

F (a) =

∫ a

−∞f(x)dx,

for all a ∈ R, where f is said to be the probability density func-tion.

There

P (X = a) = limε↓0

∫ a+ε

a−εf(x)dx = 0

In other words, distribution µ contains no point-mass compo-nents.

MTH 664 19

Discrete random variables.

X is a discrete random variable if there are countably manyvalues

a1, a2, · · · ∈ Rsuch that

µ(A) =∑j

pj δaj(A),

where pj > 0 and∑j

pj = 1.

There

F (aj)− F (aj−) = pj

and F (x)− F (x−) = 0 if x 6∈ a1, a2, . . . .

MTH 664 20

Expectation.

If∫

Ω|X(ω)| dP (ω) exists and is finite, then

E[X] =

∫Ω

X(ω) dP (ω) =

∫Rx dµ(x)

Properties:

• E[X + Y ] = E[X] + E[Y ]

• E[aX + b] = aE[X] + b for any a, b ∈ R

• If X(ω) ≤ Y (ω), ∀ω ∈ Ω , then E[X] ≤ E[Y ]

• If P

(ω ∈ Ω : X(ω) ≤ Y (ω)

)= 1, then E[X] ≤ E[Y ]

• If g(x) ∈ B, then

E[g(X)] =

∫Ω

g(X(ω)) dP (ω) =

∫Rg(x) dµ(x)

MTH 664 21

Jensen’s inequality.

A function ϕ(x) is said to be convex over an interval I, thedomain of the function, if

ϕ(λa+ (1− λ)b) ≤ λϕ(a) + (1− λ)ϕ(b)

for all λ ∈ [0,1] and all real a and b in I.

Jensen’s inequality: Suppose E[X] <∞ and ϕ is convex. Then

ϕ(E[X]) ≤ E[ϕ(X)]

Proof. Let ρ = E[X]. There is a line `(x) = ax+ b such that

`(x) ≤ ϕ(x) and `(ρ) = ϕ(ρ)

Then

ϕ(ρ) = `(ρ) = E[`(X)] ≤ E[ϕ(X)]

MTH 664 22

Jensen’s inequality.

Jensen’s inequality: Suppose E[X] <∞ and ϕ is convex. Then

ϕ(E[X]) ≤ E[ϕ(X)]

Examples:

• E[X2] ≥(E[X]

)2

• E[eaX] ≥ eaE[X]

• If X > 0 then E[X3] ≥(E[X]

)3

• If X > 0 then E[X · ln(X)] ≥ E[X] · ln(E[X])as ϕ(x) = x ln(x) is convex for x > 0.

• If E[X] <∞ and ϕ is concave, then E[ϕ(X)] ≤ ϕ(E[X])

• If X > 0 then E[ln(X)] ≤ ln(E[X]) as ϕ(x) = ln(x) isconcave for x > 0.

MTH 664 Lectures 6, 7, & 8

Documents

Transcript of MTH 664 Lectures 6, 7, & 8