18.175: Lecture 1 .1in Probability spaces, distributions, random...

50
Probability spaces and σ-algebras Distributions on R Extension theorems 18.175: Lecture 1 Probability spaces, distributions, random variables, measure theory Scott Sheffield MIT 18.175 Lecture 1

Transcript of 18.175: Lecture 1 .1in Probability spaces, distributions, random...

Page 1: 18.175: Lecture 1 .1in Probability spaces, distributions, random ...math.mit.edu/~sheffield/2016175/Lecture1.pdf · Probability spaces and ˙-algebras Distributions on R Extension

Probability spaces and σ-algebrasDistributions on R

Extension theorems

18.175: Lecture 1

Probability spaces, distributions, randomvariables, measure theory

Scott Sheffield

MIT

18.175 Lecture 1

Page 2: 18.175: Lecture 1 .1in Probability spaces, distributions, random ...math.mit.edu/~sheffield/2016175/Lecture1.pdf · Probability spaces and ˙-algebras Distributions on R Extension

Probability spaces and σ-algebrasDistributions on R

Extension theorems

Outline

Probability spaces and σ-algebras

Distributions on R

Extension theorems

18.175 Lecture 1

Page 3: 18.175: Lecture 1 .1in Probability spaces, distributions, random ...math.mit.edu/~sheffield/2016175/Lecture1.pdf · Probability spaces and ˙-algebras Distributions on R Extension

Probability spaces and σ-algebrasDistributions on R

Extension theorems

Outline

Probability spaces and σ-algebras

Distributions on R

Extension theorems

18.175 Lecture 1

Page 4: 18.175: Lecture 1 .1in Probability spaces, distributions, random ...math.mit.edu/~sheffield/2016175/Lecture1.pdf · Probability spaces and ˙-algebras Distributions on R Extension

Probability spaces and σ-algebrasDistributions on R

Extension theorems

Probability space notation

I Probability space is triple (Ω,F ,P) where Ω is samplespace, F is set of events (the σ-algebra) and P : F → [0, 1] isthe probability function.

I σ-algebra is collection of subsets closed undercomplementation and countable unions. Call (Ω,F) ameasure space.

I Measure is function µ : F → R satisfying µ(A) ≥ µ(∅) = 0for all A ∈ F and countable additivity: µ(∪iAi ) =

∑i µ(Ai )

for disjoint Ai .

I Measure µ is probability measure if µ(Ω) = 1.

18.175 Lecture 1

Page 5: 18.175: Lecture 1 .1in Probability spaces, distributions, random ...math.mit.edu/~sheffield/2016175/Lecture1.pdf · Probability spaces and ˙-algebras Distributions on R Extension

Probability spaces and σ-algebrasDistributions on R

Extension theorems

Probability space notation

I Probability space is triple (Ω,F ,P) where Ω is samplespace, F is set of events (the σ-algebra) and P : F → [0, 1] isthe probability function.

I σ-algebra is collection of subsets closed undercomplementation and countable unions. Call (Ω,F) ameasure space.

I Measure is function µ : F → R satisfying µ(A) ≥ µ(∅) = 0for all A ∈ F and countable additivity: µ(∪iAi ) =

∑i µ(Ai )

for disjoint Ai .

I Measure µ is probability measure if µ(Ω) = 1.

18.175 Lecture 1

Page 6: 18.175: Lecture 1 .1in Probability spaces, distributions, random ...math.mit.edu/~sheffield/2016175/Lecture1.pdf · Probability spaces and ˙-algebras Distributions on R Extension

Probability spaces and σ-algebrasDistributions on R

Extension theorems

Probability space notation

I Probability space is triple (Ω,F ,P) where Ω is samplespace, F is set of events (the σ-algebra) and P : F → [0, 1] isthe probability function.

I σ-algebra is collection of subsets closed undercomplementation and countable unions. Call (Ω,F) ameasure space.

I Measure is function µ : F → R satisfying µ(A) ≥ µ(∅) = 0for all A ∈ F and countable additivity: µ(∪iAi ) =

∑i µ(Ai )

for disjoint Ai .

I Measure µ is probability measure if µ(Ω) = 1.

18.175 Lecture 1

Page 7: 18.175: Lecture 1 .1in Probability spaces, distributions, random ...math.mit.edu/~sheffield/2016175/Lecture1.pdf · Probability spaces and ˙-algebras Distributions on R Extension

Probability spaces and σ-algebrasDistributions on R

Extension theorems

Probability space notation

I Probability space is triple (Ω,F ,P) where Ω is samplespace, F is set of events (the σ-algebra) and P : F → [0, 1] isthe probability function.

I σ-algebra is collection of subsets closed undercomplementation and countable unions. Call (Ω,F) ameasure space.

I Measure is function µ : F → R satisfying µ(A) ≥ µ(∅) = 0for all A ∈ F and countable additivity: µ(∪iAi ) =

∑i µ(Ai )

for disjoint Ai .

I Measure µ is probability measure if µ(Ω) = 1.

18.175 Lecture 1

Page 8: 18.175: Lecture 1 .1in Probability spaces, distributions, random ...math.mit.edu/~sheffield/2016175/Lecture1.pdf · Probability spaces and ˙-algebras Distributions on R Extension

Probability spaces and σ-algebrasDistributions on R

Extension theorems

Basic consequences of definitions

I monotonicity: A ⊂ B implies µ(A) ⊂ µ(B)

I subadditivity: A ⊂ ∪∞m=1Am implies µ(A) ≤∑∞

m=1 µ(Am).

I continuity from below: measures of sets Ai in increasingsequence converge to measure of limit ∪iAi

I continuity from above: measures of sets Ai in decreasingsequence converge to measure of intersection ∩iAi

18.175 Lecture 1

Page 9: 18.175: Lecture 1 .1in Probability spaces, distributions, random ...math.mit.edu/~sheffield/2016175/Lecture1.pdf · Probability spaces and ˙-algebras Distributions on R Extension

Probability spaces and σ-algebrasDistributions on R

Extension theorems

Basic consequences of definitions

I monotonicity: A ⊂ B implies µ(A) ⊂ µ(B)

I subadditivity: A ⊂ ∪∞m=1Am implies µ(A) ≤∑∞

m=1 µ(Am).

I continuity from below: measures of sets Ai in increasingsequence converge to measure of limit ∪iAi

I continuity from above: measures of sets Ai in decreasingsequence converge to measure of intersection ∩iAi

18.175 Lecture 1

Page 10: 18.175: Lecture 1 .1in Probability spaces, distributions, random ...math.mit.edu/~sheffield/2016175/Lecture1.pdf · Probability spaces and ˙-algebras Distributions on R Extension

Probability spaces and σ-algebrasDistributions on R

Extension theorems

Basic consequences of definitions

I monotonicity: A ⊂ B implies µ(A) ⊂ µ(B)

I subadditivity: A ⊂ ∪∞m=1Am implies µ(A) ≤∑∞

m=1 µ(Am).

I continuity from below: measures of sets Ai in increasingsequence converge to measure of limit ∪iAi

I continuity from above: measures of sets Ai in decreasingsequence converge to measure of intersection ∩iAi

18.175 Lecture 1

Page 11: 18.175: Lecture 1 .1in Probability spaces, distributions, random ...math.mit.edu/~sheffield/2016175/Lecture1.pdf · Probability spaces and ˙-algebras Distributions on R Extension

Probability spaces and σ-algebrasDistributions on R

Extension theorems

Basic consequences of definitions

I monotonicity: A ⊂ B implies µ(A) ⊂ µ(B)

I subadditivity: A ⊂ ∪∞m=1Am implies µ(A) ≤∑∞

m=1 µ(Am).

I continuity from below: measures of sets Ai in increasingsequence converge to measure of limit ∪iAi

I continuity from above: measures of sets Ai in decreasingsequence converge to measure of intersection ∩iAi

18.175 Lecture 1

Page 12: 18.175: Lecture 1 .1in Probability spaces, distributions, random ...math.mit.edu/~sheffield/2016175/Lecture1.pdf · Probability spaces and ˙-algebras Distributions on R Extension

Probability spaces and σ-algebrasDistributions on R

Extension theorems

Why can’t σ-algebra be all subsets of Ω?

I Uniform probability measure on [0, 1) should satisfytranslation invariance: If B and a horizontal translation of Bare both subsets [0, 1), their probabilities should be equal.

I Consider wrap-around translations τr (x) = (x + r) mod 1.I By translation invariance, τr (B) has same probability as B.I Call x , y “equivalent modulo rationals” if x − y is rational

(e.g., x = π − 3 and y = π − 9/4). An equivalence class isthe set of points in [0, 1) equivalent to some given point.

I There are uncountably many of these classes.I Let A ⊂ [0, 1) contain one point from each class. For each

x ∈ [0, 1), there is one a ∈ A such that r = x − a is rational.I Then each x in [0, 1) lies in τr (A) for one rational r ∈ [0, 1).I Thus [0, 1) = ∪τr (A) as r ranges over rationals in [0, 1).I If P(A) = 0, then P(S) =

∑r P(τr (A)) = 0. If P(A) > 0 then

P(S) =∑

r P(τr (A)) =∞. Contradicts P(S) = 1 axiom.

18.175 Lecture 1

Page 13: 18.175: Lecture 1 .1in Probability spaces, distributions, random ...math.mit.edu/~sheffield/2016175/Lecture1.pdf · Probability spaces and ˙-algebras Distributions on R Extension

Probability spaces and σ-algebrasDistributions on R

Extension theorems

Why can’t σ-algebra be all subsets of Ω?

I Uniform probability measure on [0, 1) should satisfytranslation invariance: If B and a horizontal translation of Bare both subsets [0, 1), their probabilities should be equal.

I Consider wrap-around translations τr (x) = (x + r) mod 1.

I By translation invariance, τr (B) has same probability as B.I Call x , y “equivalent modulo rationals” if x − y is rational

(e.g., x = π − 3 and y = π − 9/4). An equivalence class isthe set of points in [0, 1) equivalent to some given point.

I There are uncountably many of these classes.I Let A ⊂ [0, 1) contain one point from each class. For each

x ∈ [0, 1), there is one a ∈ A such that r = x − a is rational.I Then each x in [0, 1) lies in τr (A) for one rational r ∈ [0, 1).I Thus [0, 1) = ∪τr (A) as r ranges over rationals in [0, 1).I If P(A) = 0, then P(S) =

∑r P(τr (A)) = 0. If P(A) > 0 then

P(S) =∑

r P(τr (A)) =∞. Contradicts P(S) = 1 axiom.

18.175 Lecture 1

Page 14: 18.175: Lecture 1 .1in Probability spaces, distributions, random ...math.mit.edu/~sheffield/2016175/Lecture1.pdf · Probability spaces and ˙-algebras Distributions on R Extension

Probability spaces and σ-algebrasDistributions on R

Extension theorems

Why can’t σ-algebra be all subsets of Ω?

I Uniform probability measure on [0, 1) should satisfytranslation invariance: If B and a horizontal translation of Bare both subsets [0, 1), their probabilities should be equal.

I Consider wrap-around translations τr (x) = (x + r) mod 1.I By translation invariance, τr (B) has same probability as B.

I Call x , y “equivalent modulo rationals” if x − y is rational(e.g., x = π − 3 and y = π − 9/4). An equivalence class isthe set of points in [0, 1) equivalent to some given point.

I There are uncountably many of these classes.I Let A ⊂ [0, 1) contain one point from each class. For each

x ∈ [0, 1), there is one a ∈ A such that r = x − a is rational.I Then each x in [0, 1) lies in τr (A) for one rational r ∈ [0, 1).I Thus [0, 1) = ∪τr (A) as r ranges over rationals in [0, 1).I If P(A) = 0, then P(S) =

∑r P(τr (A)) = 0. If P(A) > 0 then

P(S) =∑

r P(τr (A)) =∞. Contradicts P(S) = 1 axiom.

18.175 Lecture 1

Page 15: 18.175: Lecture 1 .1in Probability spaces, distributions, random ...math.mit.edu/~sheffield/2016175/Lecture1.pdf · Probability spaces and ˙-algebras Distributions on R Extension

Probability spaces and σ-algebrasDistributions on R

Extension theorems

Why can’t σ-algebra be all subsets of Ω?

I Uniform probability measure on [0, 1) should satisfytranslation invariance: If B and a horizontal translation of Bare both subsets [0, 1), their probabilities should be equal.

I Consider wrap-around translations τr (x) = (x + r) mod 1.I By translation invariance, τr (B) has same probability as B.I Call x , y “equivalent modulo rationals” if x − y is rational

(e.g., x = π − 3 and y = π − 9/4). An equivalence class isthe set of points in [0, 1) equivalent to some given point.

I There are uncountably many of these classes.I Let A ⊂ [0, 1) contain one point from each class. For each

x ∈ [0, 1), there is one a ∈ A such that r = x − a is rational.I Then each x in [0, 1) lies in τr (A) for one rational r ∈ [0, 1).I Thus [0, 1) = ∪τr (A) as r ranges over rationals in [0, 1).I If P(A) = 0, then P(S) =

∑r P(τr (A)) = 0. If P(A) > 0 then

P(S) =∑

r P(τr (A)) =∞. Contradicts P(S) = 1 axiom.

18.175 Lecture 1

Page 16: 18.175: Lecture 1 .1in Probability spaces, distributions, random ...math.mit.edu/~sheffield/2016175/Lecture1.pdf · Probability spaces and ˙-algebras Distributions on R Extension

Probability spaces and σ-algebrasDistributions on R

Extension theorems

Why can’t σ-algebra be all subsets of Ω?

I Uniform probability measure on [0, 1) should satisfytranslation invariance: If B and a horizontal translation of Bare both subsets [0, 1), their probabilities should be equal.

I Consider wrap-around translations τr (x) = (x + r) mod 1.I By translation invariance, τr (B) has same probability as B.I Call x , y “equivalent modulo rationals” if x − y is rational

(e.g., x = π − 3 and y = π − 9/4). An equivalence class isthe set of points in [0, 1) equivalent to some given point.

I There are uncountably many of these classes.

I Let A ⊂ [0, 1) contain one point from each class. For eachx ∈ [0, 1), there is one a ∈ A such that r = x − a is rational.

I Then each x in [0, 1) lies in τr (A) for one rational r ∈ [0, 1).I Thus [0, 1) = ∪τr (A) as r ranges over rationals in [0, 1).I If P(A) = 0, then P(S) =

∑r P(τr (A)) = 0. If P(A) > 0 then

P(S) =∑

r P(τr (A)) =∞. Contradicts P(S) = 1 axiom.

18.175 Lecture 1

Page 17: 18.175: Lecture 1 .1in Probability spaces, distributions, random ...math.mit.edu/~sheffield/2016175/Lecture1.pdf · Probability spaces and ˙-algebras Distributions on R Extension

Probability spaces and σ-algebrasDistributions on R

Extension theorems

Why can’t σ-algebra be all subsets of Ω?

I Uniform probability measure on [0, 1) should satisfytranslation invariance: If B and a horizontal translation of Bare both subsets [0, 1), their probabilities should be equal.

I Consider wrap-around translations τr (x) = (x + r) mod 1.I By translation invariance, τr (B) has same probability as B.I Call x , y “equivalent modulo rationals” if x − y is rational

(e.g., x = π − 3 and y = π − 9/4). An equivalence class isthe set of points in [0, 1) equivalent to some given point.

I There are uncountably many of these classes.I Let A ⊂ [0, 1) contain one point from each class. For each

x ∈ [0, 1), there is one a ∈ A such that r = x − a is rational.

I Then each x in [0, 1) lies in τr (A) for one rational r ∈ [0, 1).I Thus [0, 1) = ∪τr (A) as r ranges over rationals in [0, 1).I If P(A) = 0, then P(S) =

∑r P(τr (A)) = 0. If P(A) > 0 then

P(S) =∑

r P(τr (A)) =∞. Contradicts P(S) = 1 axiom.

18.175 Lecture 1

Page 18: 18.175: Lecture 1 .1in Probability spaces, distributions, random ...math.mit.edu/~sheffield/2016175/Lecture1.pdf · Probability spaces and ˙-algebras Distributions on R Extension

Probability spaces and σ-algebrasDistributions on R

Extension theorems

Why can’t σ-algebra be all subsets of Ω?

I Uniform probability measure on [0, 1) should satisfytranslation invariance: If B and a horizontal translation of Bare both subsets [0, 1), their probabilities should be equal.

I Consider wrap-around translations τr (x) = (x + r) mod 1.I By translation invariance, τr (B) has same probability as B.I Call x , y “equivalent modulo rationals” if x − y is rational

(e.g., x = π − 3 and y = π − 9/4). An equivalence class isthe set of points in [0, 1) equivalent to some given point.

I There are uncountably many of these classes.I Let A ⊂ [0, 1) contain one point from each class. For each

x ∈ [0, 1), there is one a ∈ A such that r = x − a is rational.I Then each x in [0, 1) lies in τr (A) for one rational r ∈ [0, 1).

I Thus [0, 1) = ∪τr (A) as r ranges over rationals in [0, 1).I If P(A) = 0, then P(S) =

∑r P(τr (A)) = 0. If P(A) > 0 then

P(S) =∑

r P(τr (A)) =∞. Contradicts P(S) = 1 axiom.

18.175 Lecture 1

Page 19: 18.175: Lecture 1 .1in Probability spaces, distributions, random ...math.mit.edu/~sheffield/2016175/Lecture1.pdf · Probability spaces and ˙-algebras Distributions on R Extension

Probability spaces and σ-algebrasDistributions on R

Extension theorems

Why can’t σ-algebra be all subsets of Ω?

I Uniform probability measure on [0, 1) should satisfytranslation invariance: If B and a horizontal translation of Bare both subsets [0, 1), their probabilities should be equal.

I Consider wrap-around translations τr (x) = (x + r) mod 1.I By translation invariance, τr (B) has same probability as B.I Call x , y “equivalent modulo rationals” if x − y is rational

(e.g., x = π − 3 and y = π − 9/4). An equivalence class isthe set of points in [0, 1) equivalent to some given point.

I There are uncountably many of these classes.I Let A ⊂ [0, 1) contain one point from each class. For each

x ∈ [0, 1), there is one a ∈ A such that r = x − a is rational.I Then each x in [0, 1) lies in τr (A) for one rational r ∈ [0, 1).I Thus [0, 1) = ∪τr (A) as r ranges over rationals in [0, 1).

I If P(A) = 0, then P(S) =∑

r P(τr (A)) = 0. If P(A) > 0 thenP(S) =

∑r P(τr (A)) =∞. Contradicts P(S) = 1 axiom.

18.175 Lecture 1

Page 20: 18.175: Lecture 1 .1in Probability spaces, distributions, random ...math.mit.edu/~sheffield/2016175/Lecture1.pdf · Probability spaces and ˙-algebras Distributions on R Extension

Probability spaces and σ-algebrasDistributions on R

Extension theorems

Why can’t σ-algebra be all subsets of Ω?

I Uniform probability measure on [0, 1) should satisfytranslation invariance: If B and a horizontal translation of Bare both subsets [0, 1), their probabilities should be equal.

I Consider wrap-around translations τr (x) = (x + r) mod 1.I By translation invariance, τr (B) has same probability as B.I Call x , y “equivalent modulo rationals” if x − y is rational

(e.g., x = π − 3 and y = π − 9/4). An equivalence class isthe set of points in [0, 1) equivalent to some given point.

I There are uncountably many of these classes.I Let A ⊂ [0, 1) contain one point from each class. For each

x ∈ [0, 1), there is one a ∈ A such that r = x − a is rational.I Then each x in [0, 1) lies in τr (A) for one rational r ∈ [0, 1).I Thus [0, 1) = ∪τr (A) as r ranges over rationals in [0, 1).I If P(A) = 0, then P(S) =

∑r P(τr (A)) = 0. If P(A) > 0 then

P(S) =∑

r P(τr (A)) =∞. Contradicts P(S) = 1 axiom.18.175 Lecture 1

Page 21: 18.175: Lecture 1 .1in Probability spaces, distributions, random ...math.mit.edu/~sheffield/2016175/Lecture1.pdf · Probability spaces and ˙-algebras Distributions on R Extension

Probability spaces and σ-algebrasDistributions on R

Extension theorems

Three ways to get around this

I 1. Re-examine axioms of mathematics: the very existenceof a set A with one element from each equivalence class isconsequence of so-called axiom of choice. Removing thataxiom makes paradox goes away, since one can just suppose(pretend?) these kinds of sets don’t exist.

I 2. Re-examine axioms of probability: Replace countableadditivity with finite additivity? (Look up Banach-Tarski.)

I 3. Keep the axiom of choice and countable additivity butdon’t define probabilities of all sets: Restrict attention tosome σ-algebra of measurable sets.

I Most mainstream probability and analysis takes the thirdapproach. But good to be aware of alternatives (e.g., axiomof determinacy which implies that all sets are Lebesguemeasurable).

18.175 Lecture 1

Page 22: 18.175: Lecture 1 .1in Probability spaces, distributions, random ...math.mit.edu/~sheffield/2016175/Lecture1.pdf · Probability spaces and ˙-algebras Distributions on R Extension

Probability spaces and σ-algebrasDistributions on R

Extension theorems

Three ways to get around this

I 1. Re-examine axioms of mathematics: the very existenceof a set A with one element from each equivalence class isconsequence of so-called axiom of choice. Removing thataxiom makes paradox goes away, since one can just suppose(pretend?) these kinds of sets don’t exist.

I 2. Re-examine axioms of probability: Replace countableadditivity with finite additivity? (Look up Banach-Tarski.)

I 3. Keep the axiom of choice and countable additivity butdon’t define probabilities of all sets: Restrict attention tosome σ-algebra of measurable sets.

I Most mainstream probability and analysis takes the thirdapproach. But good to be aware of alternatives (e.g., axiomof determinacy which implies that all sets are Lebesguemeasurable).

18.175 Lecture 1

Page 23: 18.175: Lecture 1 .1in Probability spaces, distributions, random ...math.mit.edu/~sheffield/2016175/Lecture1.pdf · Probability spaces and ˙-algebras Distributions on R Extension

Probability spaces and σ-algebrasDistributions on R

Extension theorems

Three ways to get around this

I 1. Re-examine axioms of mathematics: the very existenceof a set A with one element from each equivalence class isconsequence of so-called axiom of choice. Removing thataxiom makes paradox goes away, since one can just suppose(pretend?) these kinds of sets don’t exist.

I 2. Re-examine axioms of probability: Replace countableadditivity with finite additivity? (Look up Banach-Tarski.)

I 3. Keep the axiom of choice and countable additivity butdon’t define probabilities of all sets: Restrict attention tosome σ-algebra of measurable sets.

I Most mainstream probability and analysis takes the thirdapproach. But good to be aware of alternatives (e.g., axiomof determinacy which implies that all sets are Lebesguemeasurable).

18.175 Lecture 1

Page 24: 18.175: Lecture 1 .1in Probability spaces, distributions, random ...math.mit.edu/~sheffield/2016175/Lecture1.pdf · Probability spaces and ˙-algebras Distributions on R Extension

Probability spaces and σ-algebrasDistributions on R

Extension theorems

Three ways to get around this

I 1. Re-examine axioms of mathematics: the very existenceof a set A with one element from each equivalence class isconsequence of so-called axiom of choice. Removing thataxiom makes paradox goes away, since one can just suppose(pretend?) these kinds of sets don’t exist.

I 2. Re-examine axioms of probability: Replace countableadditivity with finite additivity? (Look up Banach-Tarski.)

I 3. Keep the axiom of choice and countable additivity butdon’t define probabilities of all sets: Restrict attention tosome σ-algebra of measurable sets.

I Most mainstream probability and analysis takes the thirdapproach. But good to be aware of alternatives (e.g., axiomof determinacy which implies that all sets are Lebesguemeasurable).

18.175 Lecture 1

Page 25: 18.175: Lecture 1 .1in Probability spaces, distributions, random ...math.mit.edu/~sheffield/2016175/Lecture1.pdf · Probability spaces and ˙-algebras Distributions on R Extension

Probability spaces and σ-algebrasDistributions on R

Extension theorems

Borel σ-algebra

I The Borel σ-algebra B on a topological space is the smallestσ-algebra containing all open sets. In the case of R, it is thesmallest σ-algebra containing all open intervals.

I Say that B is “generated” by the collection of open intervals.

18.175 Lecture 1

Page 26: 18.175: Lecture 1 .1in Probability spaces, distributions, random ...math.mit.edu/~sheffield/2016175/Lecture1.pdf · Probability spaces and ˙-algebras Distributions on R Extension

Probability spaces and σ-algebrasDistributions on R

Extension theorems

Borel σ-algebra

I The Borel σ-algebra B on a topological space is the smallestσ-algebra containing all open sets. In the case of R, it is thesmallest σ-algebra containing all open intervals.

I Say that B is “generated” by the collection of open intervals.

18.175 Lecture 1

Page 27: 18.175: Lecture 1 .1in Probability spaces, distributions, random ...math.mit.edu/~sheffield/2016175/Lecture1.pdf · Probability spaces and ˙-algebras Distributions on R Extension

Probability spaces and σ-algebrasDistributions on R

Extension theorems

Outline

Probability spaces and σ-algebras

Distributions on R

Extension theorems

18.175 Lecture 1

Page 28: 18.175: Lecture 1 .1in Probability spaces, distributions, random ...math.mit.edu/~sheffield/2016175/Lecture1.pdf · Probability spaces and ˙-algebras Distributions on R Extension

Probability spaces and σ-algebrasDistributions on R

Extension theorems

Outline

Probability spaces and σ-algebras

Distributions on R

Extension theorems

18.175 Lecture 1

Page 29: 18.175: Lecture 1 .1in Probability spaces, distributions, random ...math.mit.edu/~sheffield/2016175/Lecture1.pdf · Probability spaces and ˙-algebras Distributions on R Extension

Probability spaces and σ-algebrasDistributions on R

Extension theorems

How do we produce measures on R?

I Write F (a) = P((−∞, a]

).

I Theorem: for each right continuous, non-decreasing functionF , tending to 0 at −∞ and to 1 at ∞, there is a uniquemeasure defined on the Borel sets of R withP((a, b]) = F (b)− F (a).

I If we’re given such a function F , then we know how tocompute the measure of any set of the form (a, b].

I We would like to extend the measure defined for these subsetsto a measure defined for the whole σ algebra generated bythese subsets.

I Seems clear how to define measure of countable union ofdisjoint intervals of the form (a, b] (just using countableadditivity). But are we confident we can extend the definitionto all Borel measurable sets in a consistent way?

18.175 Lecture 1

Page 30: 18.175: Lecture 1 .1in Probability spaces, distributions, random ...math.mit.edu/~sheffield/2016175/Lecture1.pdf · Probability spaces and ˙-algebras Distributions on R Extension

Probability spaces and σ-algebrasDistributions on R

Extension theorems

How do we produce measures on R?

I Write F (a) = P((−∞, a]

).

I Theorem: for each right continuous, non-decreasing functionF , tending to 0 at −∞ and to 1 at ∞, there is a uniquemeasure defined on the Borel sets of R withP((a, b]) = F (b)− F (a).

I If we’re given such a function F , then we know how tocompute the measure of any set of the form (a, b].

I We would like to extend the measure defined for these subsetsto a measure defined for the whole σ algebra generated bythese subsets.

I Seems clear how to define measure of countable union ofdisjoint intervals of the form (a, b] (just using countableadditivity). But are we confident we can extend the definitionto all Borel measurable sets in a consistent way?

18.175 Lecture 1

Page 31: 18.175: Lecture 1 .1in Probability spaces, distributions, random ...math.mit.edu/~sheffield/2016175/Lecture1.pdf · Probability spaces and ˙-algebras Distributions on R Extension

Probability spaces and σ-algebrasDistributions on R

Extension theorems

How do we produce measures on R?

I Write F (a) = P((−∞, a]

).

I Theorem: for each right continuous, non-decreasing functionF , tending to 0 at −∞ and to 1 at ∞, there is a uniquemeasure defined on the Borel sets of R withP((a, b]) = F (b)− F (a).

I If we’re given such a function F , then we know how tocompute the measure of any set of the form (a, b].

I We would like to extend the measure defined for these subsetsto a measure defined for the whole σ algebra generated bythese subsets.

I Seems clear how to define measure of countable union ofdisjoint intervals of the form (a, b] (just using countableadditivity). But are we confident we can extend the definitionto all Borel measurable sets in a consistent way?

18.175 Lecture 1

Page 32: 18.175: Lecture 1 .1in Probability spaces, distributions, random ...math.mit.edu/~sheffield/2016175/Lecture1.pdf · Probability spaces and ˙-algebras Distributions on R Extension

Probability spaces and σ-algebrasDistributions on R

Extension theorems

How do we produce measures on R?

I Write F (a) = P((−∞, a]

).

I Theorem: for each right continuous, non-decreasing functionF , tending to 0 at −∞ and to 1 at ∞, there is a uniquemeasure defined on the Borel sets of R withP((a, b]) = F (b)− F (a).

I If we’re given such a function F , then we know how tocompute the measure of any set of the form (a, b].

I We would like to extend the measure defined for these subsetsto a measure defined for the whole σ algebra generated bythese subsets.

I Seems clear how to define measure of countable union ofdisjoint intervals of the form (a, b] (just using countableadditivity). But are we confident we can extend the definitionto all Borel measurable sets in a consistent way?

18.175 Lecture 1

Page 33: 18.175: Lecture 1 .1in Probability spaces, distributions, random ...math.mit.edu/~sheffield/2016175/Lecture1.pdf · Probability spaces and ˙-algebras Distributions on R Extension

Probability spaces and σ-algebrasDistributions on R

Extension theorems

How do we produce measures on R?

I Write F (a) = P((−∞, a]

).

I Theorem: for each right continuous, non-decreasing functionF , tending to 0 at −∞ and to 1 at ∞, there is a uniquemeasure defined on the Borel sets of R withP((a, b]) = F (b)− F (a).

I If we’re given such a function F , then we know how tocompute the measure of any set of the form (a, b].

I We would like to extend the measure defined for these subsetsto a measure defined for the whole σ algebra generated bythese subsets.

I Seems clear how to define measure of countable union ofdisjoint intervals of the form (a, b] (just using countableadditivity). But are we confident we can extend the definitionto all Borel measurable sets in a consistent way?

18.175 Lecture 1

Page 34: 18.175: Lecture 1 .1in Probability spaces, distributions, random ...math.mit.edu/~sheffield/2016175/Lecture1.pdf · Probability spaces and ˙-algebras Distributions on R Extension

Probability spaces and σ-algebrasDistributions on R

Extension theorems

Outline

Probability spaces and σ-algebras

Distributions on R

Extension theorems

18.175 Lecture 1

Page 35: 18.175: Lecture 1 .1in Probability spaces, distributions, random ...math.mit.edu/~sheffield/2016175/Lecture1.pdf · Probability spaces and ˙-algebras Distributions on R Extension

Probability spaces and σ-algebrasDistributions on R

Extension theorems

Outline

Probability spaces and σ-algebras

Distributions on R

Extension theorems

18.175 Lecture 1

Page 36: 18.175: Lecture 1 .1in Probability spaces, distributions, random ...math.mit.edu/~sheffield/2016175/Lecture1.pdf · Probability spaces and ˙-algebras Distributions on R Extension

Probability spaces and σ-algebrasDistributions on R

Extension theorems

Algebras and semi-algebras

I algebra: collection A of sets closed under finite unions andcomplementation.

I measure on algebra: Have µ(A) ≥ µ(∅) = 0 for all A in A,and for disjoint Ai with union in A we haveµ(∪∞i=1Ai ) =

∑∞i=1 µ(Ai ) (countable additivity).

I Measure µ on A is σ-finite if exists countable collectionAn ∈ A with µ(An) <∞ and ∪An = Ω.

I semi-algebra: collection S of sets closed under intersectionand such that S ∈ S implies that Sc is a finite disjoint unionof sets in S. (Example: empty set plus sets of form(a1, b1]× . . .× (ad , bd ] ∈ Rd .)

I One lemma: If S is a semialgebra, then the set S of finitedisjoint unions of sets in S is an algebra, called the algebragenerated by S.

18.175 Lecture 1

Page 37: 18.175: Lecture 1 .1in Probability spaces, distributions, random ...math.mit.edu/~sheffield/2016175/Lecture1.pdf · Probability spaces and ˙-algebras Distributions on R Extension

Probability spaces and σ-algebrasDistributions on R

Extension theorems

Algebras and semi-algebras

I algebra: collection A of sets closed under finite unions andcomplementation.

I measure on algebra: Have µ(A) ≥ µ(∅) = 0 for all A in A,and for disjoint Ai with union in A we haveµ(∪∞i=1Ai ) =

∑∞i=1 µ(Ai ) (countable additivity).

I Measure µ on A is σ-finite if exists countable collectionAn ∈ A with µ(An) <∞ and ∪An = Ω.

I semi-algebra: collection S of sets closed under intersectionand such that S ∈ S implies that Sc is a finite disjoint unionof sets in S. (Example: empty set plus sets of form(a1, b1]× . . .× (ad , bd ] ∈ Rd .)

I One lemma: If S is a semialgebra, then the set S of finitedisjoint unions of sets in S is an algebra, called the algebragenerated by S.

18.175 Lecture 1

Page 38: 18.175: Lecture 1 .1in Probability spaces, distributions, random ...math.mit.edu/~sheffield/2016175/Lecture1.pdf · Probability spaces and ˙-algebras Distributions on R Extension

Probability spaces and σ-algebrasDistributions on R

Extension theorems

Algebras and semi-algebras

I algebra: collection A of sets closed under finite unions andcomplementation.

I measure on algebra: Have µ(A) ≥ µ(∅) = 0 for all A in A,and for disjoint Ai with union in A we haveµ(∪∞i=1Ai ) =

∑∞i=1 µ(Ai ) (countable additivity).

I Measure µ on A is σ-finite if exists countable collectionAn ∈ A with µ(An) <∞ and ∪An = Ω.

I semi-algebra: collection S of sets closed under intersectionand such that S ∈ S implies that Sc is a finite disjoint unionof sets in S. (Example: empty set plus sets of form(a1, b1]× . . .× (ad , bd ] ∈ Rd .)

I One lemma: If S is a semialgebra, then the set S of finitedisjoint unions of sets in S is an algebra, called the algebragenerated by S.

18.175 Lecture 1

Page 39: 18.175: Lecture 1 .1in Probability spaces, distributions, random ...math.mit.edu/~sheffield/2016175/Lecture1.pdf · Probability spaces and ˙-algebras Distributions on R Extension

Probability spaces and σ-algebrasDistributions on R

Extension theorems

Algebras and semi-algebras

I algebra: collection A of sets closed under finite unions andcomplementation.

I measure on algebra: Have µ(A) ≥ µ(∅) = 0 for all A in A,and for disjoint Ai with union in A we haveµ(∪∞i=1Ai ) =

∑∞i=1 µ(Ai ) (countable additivity).

I Measure µ on A is σ-finite if exists countable collectionAn ∈ A with µ(An) <∞ and ∪An = Ω.

I semi-algebra: collection S of sets closed under intersectionand such that S ∈ S implies that Sc is a finite disjoint unionof sets in S. (Example: empty set plus sets of form(a1, b1]× . . .× (ad , bd ] ∈ Rd .)

I One lemma: If S is a semialgebra, then the set S of finitedisjoint unions of sets in S is an algebra, called the algebragenerated by S.

18.175 Lecture 1

Page 40: 18.175: Lecture 1 .1in Probability spaces, distributions, random ...math.mit.edu/~sheffield/2016175/Lecture1.pdf · Probability spaces and ˙-algebras Distributions on R Extension

Probability spaces and σ-algebrasDistributions on R

Extension theorems

Algebras and semi-algebras

I algebra: collection A of sets closed under finite unions andcomplementation.

I measure on algebra: Have µ(A) ≥ µ(∅) = 0 for all A in A,and for disjoint Ai with union in A we haveµ(∪∞i=1Ai ) =

∑∞i=1 µ(Ai ) (countable additivity).

I Measure µ on A is σ-finite if exists countable collectionAn ∈ A with µ(An) <∞ and ∪An = Ω.

I semi-algebra: collection S of sets closed under intersectionand such that S ∈ S implies that Sc is a finite disjoint unionof sets in S. (Example: empty set plus sets of form(a1, b1]× . . .× (ad , bd ] ∈ Rd .)

I One lemma: If S is a semialgebra, then the set S of finitedisjoint unions of sets in S is an algebra, called the algebragenerated by S.

18.175 Lecture 1

Page 41: 18.175: Lecture 1 .1in Probability spaces, distributions, random ...math.mit.edu/~sheffield/2016175/Lecture1.pdf · Probability spaces and ˙-algebras Distributions on R Extension

Probability spaces and σ-algebrasDistributions on R

Extension theorems

π-systems and λ-systems

I Say collection of sets P is a π-system if closed underintersection.

I Say collection of sets L is a λ-system if

I Ω ∈ LI If A,B ∈ L and A ⊂ B, then B − A ∈ L.I If An ∈ L and An ↑ A then A ∈ L.

I THEOREM: If P is a π-system and L is a λ-system thatcontains P, then σ(P) ⊂ L, where σ(A) denotes smallestσ-algebra containing A.

18.175 Lecture 1

Page 42: 18.175: Lecture 1 .1in Probability spaces, distributions, random ...math.mit.edu/~sheffield/2016175/Lecture1.pdf · Probability spaces and ˙-algebras Distributions on R Extension

Probability spaces and σ-algebrasDistributions on R

Extension theorems

π-systems and λ-systems

I Say collection of sets P is a π-system if closed underintersection.

I Say collection of sets L is a λ-system if

I Ω ∈ LI If A,B ∈ L and A ⊂ B, then B − A ∈ L.I If An ∈ L and An ↑ A then A ∈ L.

I THEOREM: If P is a π-system and L is a λ-system thatcontains P, then σ(P) ⊂ L, where σ(A) denotes smallestσ-algebra containing A.

18.175 Lecture 1

Page 43: 18.175: Lecture 1 .1in Probability spaces, distributions, random ...math.mit.edu/~sheffield/2016175/Lecture1.pdf · Probability spaces and ˙-algebras Distributions on R Extension

Probability spaces and σ-algebrasDistributions on R

Extension theorems

π-systems and λ-systems

I Say collection of sets P is a π-system if closed underintersection.

I Say collection of sets L is a λ-system ifI Ω ∈ L

I If A,B ∈ L and A ⊂ B, then B − A ∈ L.I If An ∈ L and An ↑ A then A ∈ L.

I THEOREM: If P is a π-system and L is a λ-system thatcontains P, then σ(P) ⊂ L, where σ(A) denotes smallestσ-algebra containing A.

18.175 Lecture 1

Page 44: 18.175: Lecture 1 .1in Probability spaces, distributions, random ...math.mit.edu/~sheffield/2016175/Lecture1.pdf · Probability spaces and ˙-algebras Distributions on R Extension

Probability spaces and σ-algebrasDistributions on R

Extension theorems

π-systems and λ-systems

I Say collection of sets P is a π-system if closed underintersection.

I Say collection of sets L is a λ-system ifI Ω ∈ LI If A,B ∈ L and A ⊂ B, then B − A ∈ L.

I If An ∈ L and An ↑ A then A ∈ L.

I THEOREM: If P is a π-system and L is a λ-system thatcontains P, then σ(P) ⊂ L, where σ(A) denotes smallestσ-algebra containing A.

18.175 Lecture 1

Page 45: 18.175: Lecture 1 .1in Probability spaces, distributions, random ...math.mit.edu/~sheffield/2016175/Lecture1.pdf · Probability spaces and ˙-algebras Distributions on R Extension

Probability spaces and σ-algebrasDistributions on R

Extension theorems

π-systems and λ-systems

I Say collection of sets P is a π-system if closed underintersection.

I Say collection of sets L is a λ-system ifI Ω ∈ LI If A,B ∈ L and A ⊂ B, then B − A ∈ L.I If An ∈ L and An ↑ A then A ∈ L.

I THEOREM: If P is a π-system and L is a λ-system thatcontains P, then σ(P) ⊂ L, where σ(A) denotes smallestσ-algebra containing A.

18.175 Lecture 1

Page 46: 18.175: Lecture 1 .1in Probability spaces, distributions, random ...math.mit.edu/~sheffield/2016175/Lecture1.pdf · Probability spaces and ˙-algebras Distributions on R Extension

Probability spaces and σ-algebrasDistributions on R

Extension theorems

π-systems and λ-systems

I Say collection of sets P is a π-system if closed underintersection.

I Say collection of sets L is a λ-system ifI Ω ∈ LI If A,B ∈ L and A ⊂ B, then B − A ∈ L.I If An ∈ L and An ↑ A then A ∈ L.

I THEOREM: If P is a π-system and L is a λ-system thatcontains P, then σ(P) ⊂ L, where σ(A) denotes smallestσ-algebra containing A.

18.175 Lecture 1

Page 47: 18.175: Lecture 1 .1in Probability spaces, distributions, random ...math.mit.edu/~sheffield/2016175/Lecture1.pdf · Probability spaces and ˙-algebras Distributions on R Extension

Probability spaces and σ-algebrasDistributions on R

Extension theorems

Caratheeodory Extension Theorem

I Theorem: If µ is a σ-finite measure on an algebra A then µhas a unique extension to the σ algebra generated by A.

I Detailed proof is somewhat involved, but let’s take a look atit.

I We can use this extension theorem prove existence of a uniquetranslation invariant measure (Lebesgue measure) on theBorel sets of Rd that assigns unit mass to a unit cube. (Borelσ-algebra Rd is the smallest one containing all open sets ofRd . Given any space with a topology, we can define aσ-algebra this way.)

18.175 Lecture 1

Page 48: 18.175: Lecture 1 .1in Probability spaces, distributions, random ...math.mit.edu/~sheffield/2016175/Lecture1.pdf · Probability spaces and ˙-algebras Distributions on R Extension

Probability spaces and σ-algebrasDistributions on R

Extension theorems

Caratheeodory Extension Theorem

I Theorem: If µ is a σ-finite measure on an algebra A then µhas a unique extension to the σ algebra generated by A.

I Detailed proof is somewhat involved, but let’s take a look atit.

I We can use this extension theorem prove existence of a uniquetranslation invariant measure (Lebesgue measure) on theBorel sets of Rd that assigns unit mass to a unit cube. (Borelσ-algebra Rd is the smallest one containing all open sets ofRd . Given any space with a topology, we can define aσ-algebra this way.)

18.175 Lecture 1

Page 49: 18.175: Lecture 1 .1in Probability spaces, distributions, random ...math.mit.edu/~sheffield/2016175/Lecture1.pdf · Probability spaces and ˙-algebras Distributions on R Extension

Probability spaces and σ-algebrasDistributions on R

Extension theorems

Caratheeodory Extension Theorem

I Theorem: If µ is a σ-finite measure on an algebra A then µhas a unique extension to the σ algebra generated by A.

I Detailed proof is somewhat involved, but let’s take a look atit.

I We can use this extension theorem prove existence of a uniquetranslation invariant measure (Lebesgue measure) on theBorel sets of Rd that assigns unit mass to a unit cube. (Borelσ-algebra Rd is the smallest one containing all open sets ofRd . Given any space with a topology, we can define aσ-algebra this way.)

18.175 Lecture 1

Page 50: 18.175: Lecture 1 .1in Probability spaces, distributions, random ...math.mit.edu/~sheffield/2016175/Lecture1.pdf · Probability spaces and ˙-algebras Distributions on R Extension

Probability spaces and σ-algebrasDistributions on R

Extension theorems

Extension theorem for semialgebras

I Say S is semialgebra and µ is defined on S with µ(∅ = 0),such that µ is finitely additive and countably subadditive.[This means that if S ∈ S is a finite disjoint union of setsSi ∈ S then µ(S) =

∑i µ(Si ). If it is a countable disjoint

union of Si ∈ S then µ(S) ≤∑

i µ(Si ).] Then µ has a uniqueextension µ that is a measure on the algebra S generated byS. If µ is sigma-finite, then there is an extension that is ameasure on σ(S).

18.175 Lecture 1