Random Sequences
Gaurav S. Kasbekar
Dept. of Electrical Engineering
IIT Bombay
Recall
• We started with one r.v. 𝑋 on a probability space (Ω, ℱ, 𝑃)
• Then, two r.v.s 𝑋 and 𝑌 on a common probability space (Ω, ℱ, 𝑃)
• Then, a vector (𝑋1, … , 𝑋𝑛) of r.v.s on (Ω, ℱ, 𝑃)
• Next: an infinite sequence 𝑋1, 𝑋2, 𝑋3, … of r.v.s on (Ω, ℱ, 𝑃)
• We’ll study convergence of such a sequence
Motivation
• Two important results have to do with convergence of random sequences:
1) Law of Large Numbers
2) Central Limit Theorem
Law of Large Numbers • Recall motivation of the definition of 𝐸(𝑋)
• 𝑛 independent trials of experiment performed
• Average of values of 𝑋 in the 𝑛 trials used to motivate expression for 𝐸(𝑋)
• Let 𝑋1, 𝑋2, 𝑋3, … be i.i.d. with mean 𝜇
• lim𝑛→∞
𝑋1+⋯+𝑋𝑛
𝑛:
intuitively, 𝜇
• That is, letting 𝑋 𝑛 =𝑋1+⋯+𝑋𝑛
𝑛, the sequence
𝑋 1, 𝑋 2, 𝑋 3, … converges to the constant 𝜇
• To state this result formally, need to define convergence
Central Limit Theorem
• 𝑋1, 𝑋2, 𝑋3, … i.i.d. with mean 𝜇 and variance 𝜎2
• Informally, for large 𝑛, the CDF of 𝑋1 +⋯+ 𝑋𝑛 is approximately Gaussian
• That is, letting 𝑆𝑛 = 𝑋1 +⋯+ 𝑋𝑛, the distribution of 𝑆𝑛 converges to a Gaussian distribution as 𝑛 → ∞
Convergence of Real Numbers
• 𝑥1, 𝑥2, 𝑥3, …: a sequence of real numbers
• lim𝑛→∞
𝑥𝑛 = 𝑥 if:
for every 𝜖 > 0, there exists 𝑁𝜖 such that |𝑥𝑛 − 𝑥| < 𝜖 for all 𝑛 ≥ 𝑁𝜖
• E.g., limit of 𝑥𝑛 =(−1)𝑛
𝑛:
0
• E.g., limit of 𝑥𝑛 = (−1)𝑛:
does not exist (sequence oscillates)
Convergence of Random Variables
• 𝑋1, 𝑋2, 𝑋3, … r.v.s on (Ω, ℱ, 𝑃)
• Want to define convergence of this sequence
• Recall: 𝑋𝑖 is a function from Ω to ℛ
• So convergence of r.v.s similar to convergence of functions
• Simplest notion: point-wise convergence
called sure convergence in r.v. terminology
Sure Convergence
• Definition: 𝑋1, 𝑋2, 𝑋3, … converges surely to 𝑋 if for every ω ∈ Ω, lim
𝑛→∞𝑋𝑛(ω) = 𝑋(ω)
• E.g.: box initially has 𝑤 white and 𝑏 black balls
• At each step 𝑛 = 1,2,3, … one ball is drawn at random without replacement (if any left)
• 𝑋𝑛: number of white balls left after 𝑛’th draw
• Convergence behaviour of 𝑋1, 𝑋2, 𝑋3, … :
converges surely to 0
Example
• A fair coin tossed an infinite number of times
• 𝑋𝑛 = 1 if at least one of tosses 1,… , 𝑛 results in heads and 𝑋𝑛 = 0 else
• Convergence behaviour of 𝑋1, 𝑋2, 𝑋3, … :
with probability 1, converges to 1
but for ω = "𝑇𝑇𝑇… " ∈ Ω, 𝑋𝑛 ω = 0 for all 𝑛
does not converge surely to 1
• "𝑇𝑇𝑇… " is an event of probability 0
• Typically we don’t care about 0 probability events
Almost Sure Convergence
• Definition: 𝑋1, 𝑋2, 𝑋3, … converges almost surely to 𝑋 if for every ω ∈ 𝐴 ⊆ Ω, lim𝑛→∞
𝑋𝑛(ω) = 𝑋(ω), where P 𝐴 = 1
• In coin tossing example, 𝑋1, 𝑋2, 𝑋3, … converges a.s. to 1
Example
• Ω = 0,1 , ℱ = ℬ, 𝑃 𝑎, 𝑏 = 𝑏 − 𝑎, 0 ≤ 𝑎 ≤ 𝑏 ≤ 1
• 𝑋𝑛 ω = ω𝑛
• For fixed ω ∈ Ω, lim𝑛→∞
𝑋𝑛(ω) :
0, 0 ≤ ω < 11, ω = 1.
• 𝑋1, 𝑋2, 𝑋3, … converges a.s. to: 0
• Thus, one way to show a.s. convergence of 𝑋1, 𝑋2, 𝑋3, … to 𝑋:
identify 𝐴 such that lim𝑛→∞
𝑋𝑛(ω) = 𝑋(ω) for all ω ∈ 𝐴
show that P 𝐴 = 1
Almost Sure Convergence
• In several examples where intuitively 𝑋1, 𝑋2, 𝑋3, … seems to converge to 𝑋,
a.s. convergence does not hold
Example • Ω = 0,1 , ℱ = ℬ, 𝑃 𝑎, 𝑏 = 𝑏 − 𝑎, 0 ≤ 𝑎 ≤ 𝑏 ≤ 1
Ref: Hajek, Chapter 2
Example (contd.)
• lim𝑛→∞
𝑋𝑛(ω):
does not exist for any ω ∈ Ω!
• So 𝑋1, 𝑋2, 𝑋3, … does not converge a.s. to any r.v. 𝑋
• But intuitively, the sequence seems to be converging to 0
Top Related