[Springer Series in Statistics] Bayesian and Frequentist Regression Methods || Basic Large Sample...
2
Appendix G Basic Large Sample Theory We define various quantities, and state results, that are useful in various places in the book. The presentation is informal see, for example, van der Vaart (1998) for more rigour. Modes of Convergence Suppose that Y n , n ≥ 1, are all random variables defined on a probability space (Ω, A,P ) where Ω is a set (the sample space) A is a σ-algebra of subsets of Ω, and P is a probability measure. Definition. We say that Y n converges almost surely to Y , denoted Y n → a.s. Y , if Y n (ω) → Y (ω) for all ω ∈ A where P (A c )=0 (G.1) or, equivalently, if, for every > 0 P sup m≥n |Y m − Y | > → 0 as n →∞. (G.2) Definition. We say that Y n converges in probability to Y , denoted Y n → p Y , if P (|Y m − Y | >) → 0 as n →∞. (G.3) Definition. Define the distribution function of Y as F (y) = Pr(Y ≤ y). We say that Y n converges in distribution to Y , denoted Y n → d Y , or F n → F , if F n (y) → F (y) as n →∞ for each continuity point y of F. (G.4) J. Wakefield, Bayesian and Frequentist Regression Methods, Springer Series in Statistics, DOI 10.1007/978-1-4419-0925-1 19, © Springer Science+Business Media New York 2013 673
Transcript of [Springer Series in Statistics] Bayesian and Frequentist Regression Methods || Basic Large Sample...
![Page 1: [Springer Series in Statistics] Bayesian and Frequentist Regression Methods || Basic Large Sample Theory](https://reader043.fdocument.org/reader043/viewer/2022020613/575092c81a28abbf6baa53f6/html5/page/1.jpg)
Appendix GBasic Large Sample Theory
We define various quantities, and state results, that are useful in various places inthe book. The presentation is informal see, for example, van der Vaart (1998) formore rigour.
Modes of Convergence
Suppose that Yn, n ≥ 1, are all random variables defined on a probability space(Ω,A, P ) where Ω is a set (the sample space) A is a σ-algebra of subsets of Ω, andP is a probability measure.
Definition. We say that Yn converges almost surely to Y , denoted Yn →a.s. Y , if
Yn(ω) → Y (ω) for all ω ∈ A where P (Ac) = 0 (G.1)
or, equivalently, if, for every ε > 0
P
(supm≥n
|Ym − Y | > ε
)→ 0 as n → ∞. (G.2)
Definition. We say that Yn converges in probability to Y , denoted Yn →p Y , if
P (|Ym − Y | > ε) → 0 as n → ∞. (G.3)
Definition. Define the distribution function of Y as F (y) = Pr(Y ≤ y). We saythat Yn converges in distribution to Y , denoted Yn →d Y , or Fn → F , if
Fn(y) → F (y) as n → ∞ for each continuity point y of F . (G.4)
J. Wakefield, Bayesian and Frequentist Regression Methods, Springer Seriesin Statistics, DOI 10.1007/978-1-4419-0925-1 19,© Springer Science+Business Media New York 2013
673
![Page 2: [Springer Series in Statistics] Bayesian and Frequentist Regression Methods || Basic Large Sample Theory](https://reader043.fdocument.org/reader043/viewer/2022020613/575092c81a28abbf6baa53f6/html5/page/2.jpg)
674 G Basic Large Sample Theory
Limit Theorems
Proposition (Weak Law of Large Numbers). If Y1, Y2, . . . , Yn, . . . are indepen-dent and identically distributed (i.i.d.) with mean μ = E[Y ] (so E[|Y |] < ∞) thenY n →p μ.
Proposition (Strong Law of Large Numbers). If Y1, Y2, . . . , Yn, . . . are i.i.d. withmean μ = E[Y ] (so E[|Y |] < ∞) then Y n →a.s. μ.
Proposition (Central Limit Theorem). If Y1, Y2, . . . , Yn are i.i.d. with mean μ =E[Y ] and variance σ2 (so E[Y 2] < ∞), then
√n(Y n − μ) →d N(0, σ2).
Proposition (Slutsky’s Theorem). Suppose that An →p a, Bn →p b, forconstants a and b, and Yn →d Y . Then AnYn +Bn →d aY + b.
Proposition (Delta Method). Suppose√n (Yn − μ) →d Z and suppose that
g : Rp → Rk has a derivative g′ at μ (here g′ is a k × p matrix of derivatives).
Then the delta method gives the asymptotic distribution as
√n [g(Y )− g(μ)] →d g′(μ)Z.
If Z ∼ Np(0,Σ), then
√n [g(Y )− g(μ)] →d Nk [0, g
′(μ)Σg′(μ)T] .
