Problem Set 3

Sn =∑n

i=1Xi throughout.

1. Suppose Sn/n→ 0 in probability. Show that Xn/n→ 0 in probability.

Solution. |Xn|/n = |Sn − Sn−1|/n ≤ |Sn|/n+ |Sn−1|/n→ 0 in probability.

2. Suppose X1, ..., Xn, ... are iid Cauchy r.v.s. (i.e., their common density is 1/[π(1 +x2)]). Suppose bn = cnn and cn ↑ ∞. Show that Sn/bn → 0 in probability. (Hint:verify the two conditions of WLLN.)

Solution. For condition (i),

n∑i=1

P (|Xi| > bn) = nP (|X1| > bn) = 2n

∫ ∞bn

1

π(1 + x2)dx

≤ 2n

π

∫ ∞bn

1

x2dx =

2n

πbn=

2

πcn↓ 0.

For condition (ii),

n∑i=1

E(Y 2i )/b2n = nE(X2

11{|X1|≤bn})/b2n =

2n

b2n

∫ bn

0

x2

π(1 + x2)dx ≤ 2nbn

πb2n=

2

πcn↓ 0.

Moreover, an = nE(Yj) = 0 as Xi are symmetric. Hence, Sn/bn → 0 in probability.

3. Suppose X1, ..., Xn, ... are independent with mean 0 and variance 1. Show that∑∞n=1Xn/n

p converges a.e. for all p > 1/2.

Solution.∑∞

n=1 var(Xn/np) =

∑∞n=1 1/n2p <∞. Then,

∑∞n=1Xn/n

p <∞, a.e.

4. If ψ1, ..., ψn are characteristic functions, then∏n

i=1 ψi is also a characteristic function.

Solution. Let X1, ..., Xn be r.v.s such that they are independent of each other withcharacteristic functions ψ1, ..., ψn, respectively. Then, the r.v. X1 + · · · + Xn hascharacteristic function

∏nj=1 ψj, since

E(exp{it(X1 + · · ·+Xn)}) =n∏j=1

E(exp(itXj)) =n∏j=1

ψj(t).

5. Suppose X1, ..., Xn, ... are independent with mean 0 variance 1. Suppose P (|Xn| ≤2) = 1 for all n ≥ 1. Show that Sn/

√n → N(0, 1) using characteristic functions.

(Hint: mimic the proof of the CLT in 6.2, but now {Xn} are not iid).

1

Solution. All Xn, n ≥ 1 are bounded by 2, the following approx holds by the Taylorexpansion.

E(exp(itSn/√n)) =

n∏j=1

E(exp(itXj/√n) ≈

n∏j=1

E(

1 + itXj/√n− .5t2X2

j /n)

=n∏j=1

(1− .5t2/n)→ exp{−t2/2},

as n→∞. It implies Sn/√n→ N(0, 1).

Exercises

1. Suppose Xn, n = 1, 2, ... are independent such that P (Xn = cn) = P (Xn = −cn) =1/2 where 0 < cn are constants. Show that Sn/bn → 0 in L2 and in probability if∑n

i=1 c2i /b

2n → 0.

Solution.

E(S2n/b

2n) =

n∑j=1

E(X2j )/b2n =

n∑j=1

c2j/b2n → 0.

This is the L2 convergence to 0, which also implies convergence in probability.

2. Following Exercise 1, raise a counter example to show that Sn/∑n

j=1 cj 9 0 inprobability.

Solution. Suppose cn = n2n. Then |Sn−1| ≤ (n− 1)(n− 1)2n−2 ≤ n2n−1. Therefore|Sn−1|/cn ≤ 1/n → 0. Likewise,

∑n−1j=1 cj/cn → 0 and

∑nj=1 cj/cn → 1. |Sn|/cn ≥

−|Sn−1|/cn + |Xn|/cn ≥ 1− 1/n→ 1. As a result, Sn/∑n

j=1 cj 9 0.

3. Suppose X1, ..., Xn are independent with mean 0 and variance 1. Apply Kol-mogorov’s inequality to show

E( max1≤j≤n

|Sj|) ≤ 1 + n.

(Hint: E(|X|) =∫∞

0P (|X| > t)dt)

Solution.

E( max1≤j≤n

|Sj|) =

∫ ∞0

P ( max1≤j≤n

|Sj| > t)dt ≤ 1 +

∫ ∞1

P ( max1≤j≤n

|Sj| > t)dt

≤ 1 +

∫ ∞1

(var(Sn)/t2)dt = 1 + n

∫ ∞1

1/t2dt = 1 + n.

2

4. Suppose X1, ..., Xn, ... are i.i.d. with mean 1. Show that max1≤k≤n |Xk|/|Sn| → 0a.e.. (Hint: max1≤k≤n |Xk|/n ≤ max1≤k≤n |Xk|/k and |Xn|/n→ 0 a.e..)

Solution. By the SLLN, Sn/n→ 1 a.e. It suffices to show max1≤k≤n |Xk|/n→ 0 a.e..First, since Xi has finite mean, we know E|X| < ∞ and therefore

∑∞i=1 P (|Xi| >

i/ε) <∞. It follows from the Borel-Cantelli lemma that, with probability 1, Xn/n ≤ε for all large n. In other words, Xn/n→ 0 a.e.. Write

max1≤k≤n

|Xk|/n ≤ max1≤k≤M

|Xk|/n+ maxM≤k≤n

|Xk|/n

≤ max1≤k≤M

|Xk|/n maxM≤k≤n

|Xk|/k ≤ max1≤k≤M

|Xk|/n+ maxM≤k≤∞

|Xk|/k

→ maxM≤k≤∞

|Xk|/k a.e., by letting n→∞

→ 0, a.e., by letting M →∞.

5. Suppose X1, ...Xn, ... are independent with mean 0 and variance 1. Suppose cn areconstants such that cn/n

p ↓→ 0 for some 0 < p < 1/2. Show that∑n

j=1 cjXj/n→ 0a.e.

Solution.∞∑n=1

var(cnXn)/n2 =∞∑n=1

c2n/n2 ≤

∞∑n=1

( cnnp

)2 1

n2−2p<∞.

By Kolmogorov’s criterion of SLLN,∑n

j=1 cjXj/n→ 0, a.e..

6. Suppose X1, ..., Xn, ... are iid with common density f(x) = cg(x) with g(x) = x−α

if x > 1, 1 if |x| ≤ 1 and (−x)−β if x < −1. c > 0 is some constant and α > 1and β > 2. Find the a.e. limit of Sn/n in terms of α and β. What condition onα and β would ensure

∑∞n=1Xn/n < ∞ a.e.? (Hint: verify the three conditions in

Kolmogorov’s three series theorem; first make sure that α = β (why?)).

Solution.

E(Xn/n1{|Xn|/n≤1}) = (c/n)[

∫ n

1

xx−αdx+

∫ −1

−nx(−x)−βdx+

∫ 1

−1

xdx]

= (c/n)(n−α+2 − 1

−α + 2− n−β+2 − 1

−β + 2

)(If α = 2, n−α+2−1

−α+2is replaced by log n.)

∑∞n=1E(Xn/n1{|Xn|/n≤1}) < ∞ only if

α = β. Condition (ii) holds only when α = β.

For condition (i), now that α = β,∑n

P (|Xn|/n > 1) =∑n

P (|X1| > n) = 2c∑n

∫ ∞n

x−βdx = 2c∑n

n−β+1/(β−1) <∞,

3

since β > 2.

For condition (iii),∑n

var(Xn/n1{|Xn|≤n}) =∑n

(2c/n2)[

∫ n

1

x2x−βdx+

∫ 1

0

x2dx]

=∑n

(2c/n2)(n−β+3 − 1

−β + 3+ 1/3

)<∞,

since β > 2. (Here n−β+3−1−β+3

is replaced by log(n) if β = 3.

Overall, the condition on α and β is α = β to ensure∑

nXn/n <∞, a.e..

7. What condition on α and β would ensure Sn/√n→ N(0, σ2) in Exercise 6? Express

σ2 in terms of α and β.

Solution. To require the random variables Xn to be mean 0 with finite varianceimplies α = β and β > 3. By the CLT for iid r.v.s, we have Sn/

√n → N(0, σ2)

where σ2 is the variance of Xn, which is

2c[

∫ 1

0

x2dx+

∫ ∞1

x2x−βdx] = 2c[1/3 + 1/(β − 3)].

Since 1 =∫∞−∞ f(x)dx = c

∫∞−∞ g(x)dx = 2c[1 +

∫∞1x−βdx] = 2c[1 + 1/(β − 1)], it

follows that

σ2 =1/3 + 1/(β − 3)

1 + 1/(β − 1).

8. 0 < α < 1/2. Suppose X1, ...Xn, ... are independent such that P (Xn = nα+1) =1/(6n2α) = P (Xn = −nα+1) and P (Xn = 0) = 1 − 1/(3n2α). Show that Sn/bn →N(0, 1) when b2n = var(Sn). (Hint: verify the Lindeberg condition.)

Solution. Xn are mean 0 with variance σ2n = E(X2

n) = 2n2α+2/(6n2α) = n2/3.Hence, b2n = var(Sn) =

∑nj=1 j

2/3, which is at the order of n3. (To be exact,

b2n = (1/18)n(n + 1)(2n + 1).) For any fixed ε > 0, nα+1 ≤ εbn for all large n. Asa result, for large n

∑nj=1E(X2

j 1{Xj>bnε}) = 0. Therefore, the Lindeberg conditionholds, and CLT holds thereby.

9. For X1, ..., Xn, ... independent with mean 0 and finite variances, show that the Lin-deberg condition implies max{var(Xj) : 1 ≤ j ≤ n}/var(Sn)→ 0.

Solution. Let a2n = var(Sn).

max{var(Xj) : 1 ≤ j ≤ n} = max{E(X2j 1{|Xj |>εan}) + E(X2

j 1{|Xj |≤εan}); 1 ≤ j ≤ n}

≤ max{E(X2j 1{|Xj |>εan}); 1 ≤ j ≤ n}+ ε2a2

n ≤n∑j=1

E(X2j 1{|Xj |>εan}) + εa2

n.

4

When divided by a2n the first term tends to 0 by the Lindgeberg condition and the

second term is bounded by ε. The desired result follows since ε is arbitrary.

10. Show that the central limit theorem implies WLLN.

Solution. CLT implies Sn/√n → N(0, σ2) in distribution. For any x > 0,

P (Sn/√n > x) → 1 − Φ(x) where Φ is the cdf of N(0, 1). For any fixed ε > 0,

ε√n > x for large n. It follows that, for large n, P (Sn/n > ε) = P (Sn/

√n >

ε√n) > P (Sn/

√n > x) → 1 − Φ(x). As x > 0 is arbitrary, P (Sn/n > ε) → 0 as

n→∞. Analogously, P (Sn/n < −ε)→ 0. Therefore, Sn/n→ 0 in probability andthe WLLN holds.

5