Mle Testing
-
Upload
roger-asencios-nunez -
Category
Documents
-
view
236 -
download
2
description
Transcript of Mle Testing
Basic ConceptsThe Trilogy
Application: Test for Heteroskedasticity
ML Based Hypothesis Testing
Walter Sosa-Escudero
Econ 507. Econometric Analysis. Spring 2009
April 21, 2009
Walter Sosa-Escudero ML Based Hypothesis Testing
Basic ConceptsThe Trilogy
Application: Test for Heteroskedasticity
Basic Concepts
Z ∼ f(y; θ), θ ∈ Θ ⊆ <K .
θ is a vector of K parameters.
Θ is the parameter space: set of values θ can take.
A hypothesis is a statement about θ. The goal is to learnsomething about the validity of the hypothesis, based on a sample.
If H0 : θ ∈ Θ0, the hypothesis is nested if Θ0 ⊆ Θ. In general,they are restriction hypotheses (eg., β = 0, Θ = <,Θ0 = {0}).
Walter Sosa-Escudero ML Based Hypothesis Testing
Basic ConceptsThe Trilogy
Application: Test for Heteroskedasticity
A test statistic is a random variable that has one particulardistribution when H0 is true and some other one when it isfalse.
The alternative hypothesis, HA, is a subset of all the values θcan take when H0 is false.Example: K = 1, Θ = <, H0 : θ = 0, HA : θ > 0.
The rejection region is the set of all value the test statisticcan take for which the null is rejected.
A test is a test statistic combined with a rejection region.
Walter Sosa-Escudero ML Based Hypothesis Testing
Basic ConceptsThe Trilogy
Application: Test for Heteroskedasticity
The probability that the test rejects H0 when it is true (type-Ierror) is the level of the test. It depends strictly on the nullhypothesis.
The power of a test is the probability that the test rejects H0.When H0 is false, this probability depends on HA.
The type-II error is one minus the power (do not reject whenH0 is false).
In general there is a trade-off between type I and II errors: atest with level 0 (always accepts) tends to have zero power(never reject).
The classical approach (Neyman-Pearson) proposes to set alevel exogeneously and find test that maximize power for somerelevant alternative hypothesis.
A test is consistent if the power tends to one when H0 is falseand the sample size tends to infinity.
Walter Sosa-Escudero ML Based Hypothesis Testing
Basic ConceptsThe Trilogy
Application: Test for Heteroskedasticity
Non-linear Hypothesis
H0 : h(θ) = 0, h(θ) : <K → <r,
h(θ) is a vector of r restrictions on the K parameters.
In this case Θ = <k, and
Θ0 ={θ ∈ Θ | h(θ) = 0
},⇒ Θ0 ⊆ Θ
so the hypothesis is nested.
Let D(θ), be an (r × k) matrix with (i, j) element:
D(θ)ij =∂hi(θ)∂θj
and hi(θ) denotes the i−th element of h(θ), i = 1, . . . , r. We willassume ρ(D(θ)) = r
Walter Sosa-Escudero ML Based Hypothesis Testing
Basic ConceptsThe Trilogy
Application: Test for Heteroskedasticity
Example: θ = (θ1, θ2, θ3)′. Consider H0 : h(θ) = 0, with
h(θ) =[θ1 − θ2
θ2θ3 − 1
]This is equivalent to H0 : θ1 = θ2 , θ2θ3 = 1. There are r = 2restrictions on K parameters.
In this case:
D(θ) =[
1 −1 00 θ3 θ2
]
Walter Sosa-Escudero ML Based Hypothesis Testing
Basic ConceptsThe Trilogy
Application: Test for Heteroskedasticity
The nested testing problem implies two estimators.
The unrestricted MLE, θ is defined as
θ ≡ argmaxθ∈Θ
l(θ; z)
and under suitable regularity conditions, the FOC’s are:
s(θ; z) = 0
Walter Sosa-Escudero ML Based Hypothesis Testing
Basic ConceptsThe Trilogy
Application: Test for Heteroskedasticity
Similarly, the restricted MLE, θR is defined as
θ ≡ argmaxθ∈Θ0
l(θ; z)
The lagrangean function for the restricted maximization problem is
LG(θ, λ) = l(θ; z)− λ h(θ)
and the FOC’s are:
s(θR; z)−D(θR)′λ = 0h(θR) = 0
}
Walter Sosa-Escudero ML Based Hypothesis Testing
Basic ConceptsThe Trilogy
Application: Test for Heteroskedasticity
Three Tests
Assume Vn is a consistent estimator of J . Consider the followingtest statistics for H0 : h(θ0) = 0.
1 Likelihood Ratio:
LR = n 2
[l(θ; z)n− l(θR; z)
n
]2 Wald:
W = n h(θ)′[D(θ)Vn(θ)−1D(θ)′
]−1h(θ)
3 Score (or Lagrange Multiplier):
LM = n
(s(θRn
)′V −1n (θR)
(s(θRn
)
Walter Sosa-Escudero ML Based Hypothesis Testing
Basic ConceptsThe Trilogy
Application: Test for Heteroskedasticity
Result: Under all the regularity assumptions, if Zi ∼ f(z; θ0),i = 1, 2, . . . , n, under H0 : h(θ0) = 0, LR, W and LM converge indistribution to χ2(r) and they diverge to infinity underHA : h(θ0) = δ, for any constant δ 6= 0.
The three tests are asymptotically equivalent.
The three tests are consistent.
Walter Sosa-Escudero ML Based Hypothesis Testing
Basic ConceptsThe Trilogy
Application: Test for Heteroskedasticity
Intuition: θ is always consistent for θ0. θR is consistent for θ0 onlyif H0 holds. Then, under H0:
θ and θR are consistent, then l(θ)− l(θR) ∼= 0 (LR).
h(θ) ∼= 0 (W)
The ‘shadow price’ of imposing the restriction is zero: λ ∼= 0.From the first FOC:
s(θR; z)−D(θR)′λ = 0
Since ρ(H(θ)) = r, then s(θR) ∼= 0 (score or LM).
The idea is to reject H0 when LR,W or LM are large.
Walter Sosa-Escudero ML Based Hypothesis Testing
Basic ConceptsThe Trilogy
Application: Test for Heteroskedasticity
A graphical representation
Consider the following simplification
θ ∈ < (K = 1)
H0 : θ = θ0 (a simple case).
Walter Sosa-Escudero ML Based Hypothesis Testing
Basic ConceptsThe Trilogy
Application: Test for Heteroskedasticity
Asymptotic Distributions under H0
The plan:
1 We will start with the Wald test since it depends on θ: wehave already proved its asymptotic properties, in particular
√n(θ − θ0) d→ N(0, J−1)
2 LR and LM depend on θR. We only know it is consistent.Then first we need to establish its asymptotic behavior.
Walter Sosa-Escudero ML Based Hypothesis Testing
Basic ConceptsThe Trilogy
Application: Test for Heteroskedasticity
Wald Test
W = n h(θ)′[D(θ)V −1
n D(θ)′]−1
h(θ)
Take a mean value expansion of h(θ) around θ0:
h(θ) = h(θ0) +D(θ)(θ − θ0
)where θ is a mean value between θ and θ0. Again, consistencyguarantees the exactness of the approximation.Under H0 : h(θ0) = 0
√n h(θ) = D(θ)
√n(θ − θ0
)
Walter Sosa-Escudero ML Based Hypothesis Testing
Basic ConceptsThe Trilogy
Application: Test for Heteroskedasticity
By asymptotic normality of θ
√n h(θ) d→ N
(0, D(θ0)J−1D(θ0)′
)since D(θ) and D(θ)
p→ D(θ0), Vnp→ J and θ
p→ θ0, by Slutzky’stheorem: √
n h(θ) d→ N(
0, D(θ)V −1n D(θ)′
)Then, taking the normed quadratic form:
W = n h(θ)′[D(θ)V −1
n D(θ)′]−1
h(θ) d−→ χ2(r)
Walter Sosa-Escudero ML Based Hypothesis Testing
Basic ConceptsThe Trilogy
Application: Test for Heteroskedasticity
AN of the restrited MLE
To prove the other two cases, we need two additional results:
1 √n(θR − θ0) d→ N(0, V (θ0)−1)
, where V (θ0)−1 ≡ J−1 − J−1D′[DJ−1D′]−1DJ−1,D ≡ D(θ0)
2
1√nλ ' −R(θ0)−1DJ−1 s(θ0)√
n
d→ N(0, R(θ0)−1),
with R ≡ DJ−1D′.
Walter Sosa-Escudero ML Based Hypothesis Testing
Basic ConceptsThe Trilogy
Application: Test for Heteroskedasticity
Proof: from the FOC’s
1√ns(θR; z)− 1√
nD(θR)′λ = 0
√n h(θR) = 0
(1)
Walter Sosa-Escudero ML Based Hypothesis Testing
Basic ConceptsThe Trilogy
Application: Test for Heteroskedasticity
Take a mean value expansion around θ0 for both equations:
s(θR; z) = s(θ0; z) +H(θ)(θR − θ0)1√ns(θR; z) =
1√ns(θ0; z) +
√nH(θ)n
(θR − θ0)
=1√ns(θ0; z)−
√n J (θR − θ0) (Asymptotically, Why?) (2)
√n h(θR) =
√n h(θ0) +
√n D(θ)(θR − θ0)
=√n D(θ0)(θR − θ0) (Asymptotically, Why?) (3)
Walter Sosa-Escudero ML Based Hypothesis Testing
Basic ConceptsThe Trilogy
Application: Test for Heteroskedasticity
Now replace (2) and (3) in (1) and use the fact that under H0
D(θR)p→ D(θ0).
1√ns(θ0; z)−
√n J (θR − θ0)− 1√
nD′λ = 0
√n D(θR − θ0) = 0
Solving the system (do it as homework):
√n (θR − θ0) = A
1√ns(θ0; z)
with A ≡ J−1 − J−1D′R−1DJ−1
Walter Sosa-Escudero ML Based Hypothesis Testing
Basic ConceptsThe Trilogy
Application: Test for Heteroskedasticity
Note that by our asymptotic normality result:
1√ns(θ0; z) =
√ns(θ0; z)n
=√n
∑ni=1 s(θ0, yi)
n
d→ N(0, J)
Then using Slutzky’s theorem,
√n(θR − θ0) = A
1√ns(θ0; z) d→ N(0, A J A′)
which is the desired result.
We leave as homework to prove the similar result for λ.
Walter Sosa-Escudero ML Based Hypothesis Testing
Basic ConceptsThe Trilogy
Application: Test for Heteroskedasticity
LM/Score
LM =1ns(θR)′V (θR)−1s(θR)
From the FOC of the restricted MLE problem:
1√ns(θR; z)− 1√
nD(θR)′λ = 0
Replacing:
λ′√nD(θR)V (θR)−1D(θR)′
λ√n' λ′√
nR(θ0)
λ√n
d→ χ2(r),
since λ/√n
d→ N(0, R(θ)−1), and it is a normed quadratic form.
Walter Sosa-Escudero ML Based Hypothesis Testing
Basic ConceptsThe Trilogy
Application: Test for Heteroskedasticity
LR Test
LR = 2[l(θ; z)− l(θR; z)
]Consider a second order mean value expansion of (θR) around θ,under H0:
Supress dependence on z momentarily to simplify notation
l(θR) = l(θ) + s(θ)′(θR − θ
)+
12
(θR − θ
)′H(θ)
(θR − θ
)l(θR)− l(θ) =
12
(θR − θ
)′H(θ)
(θR − θ
)(Why?)
Walter Sosa-Escudero ML Based Hypothesis Testing
Basic ConceptsThe Trilogy
Application: Test for Heteroskedasticity
Replacing and dividing and multiplying by n:
LR = −√n(θR − θ
)′( 1nH(θ)
)√n(θR − θ
)'
√n(θR − θ
)′I(θ0)
√n(θR − θ
)(Why?)
Recall
√n (θ − θ0) ' J−1 s(θ0; z)√
n
√n (θR − θ0) '
(J−1 − J−1D′R−1DJ−1
) s(θ0; z)√n
Substracting both sides:
√n(θR − θ
)' −J−1D′R−1DJ−1 s(θ0)√
n= −J−1D′
λ√n
Walter Sosa-Escudero ML Based Hypothesis Testing
Basic ConceptsThe Trilogy
Application: Test for Heteroskedasticity
Replacing above:
LR ' λ′√nDJ−1JJ−1D′
λ′√n
' λ′√nDJ−1D′
λ′√n
' LMd→ χ2(r)
Walter Sosa-Escudero ML Based Hypothesis Testing
Basic ConceptsThe Trilogy
Application: Test for Heteroskedasticity
Consistency of LR, LM and W
Now HA : h(θ0) 6= 0. Assume HA : h(θ0) = δ 6= 0.
W = n h(θ)′[D(θ)V −1
n D(θ)′]−1
h(θ)
→ n δ′[D(θ0)V −1
n D(θ0)′]−1
δ →∞
Note that under HA, plim s(θR; z)/n 6= 0 (Why?). Then:
LM =1ns(θR; z)′V (θR)−1s(θR; z)
= ns(θR; z)′
nV (θR)−1 s(θR; z)′
n→∞
Walter Sosa-Escudero ML Based Hypothesis Testing
Basic ConceptsThe Trilogy
Application: Test for Heteroskedasticity
We can use a similar argument with LR
LR = 2[l(θ; z)− l(θR; z)
]= n 2
[l(θ; z)n− l(θR; z)
n
]︸ ︷︷ ︸
6=0
→∞
(Why: recall the consistency proof...)
Walter Sosa-Escudero ML Based Hypothesis Testing
Basic ConceptsThe Trilogy
Application: Test for Heteroskedasticity
Consistency
Under H0, W, LM and LR have asymptotic χ2(r) distribution.
For a level α, the acceptance region is [0, zα], where zα is the1− α quantile of χ2(r), a finite number.
Hence, under our HA, W, LM and LR →∞, hence lie outsidethe acceptance region wpt1: they are consistent.
Walter Sosa-Escudero ML Based Hypothesis Testing
Basic ConceptsThe Trilogy
Application: Test for Heteroskedasticity
An application: A test for heteroskedasticity
Recall that the Breusch/Pagan test for heteroskedasticity is basedon the following steps:
1 Estimate by OLS, and save squared residuals e2i .
2 Regresss e2i on the Zik variables, k = 2, . . . , p and get (ESS).
The test statistic is:
12ESS ∼ χ2(p− 1) ∼ χ2(p)
under H0, asymptotically. We reject if it is too large.
This is an LM based test. The goal is to derive it from basicprinciples (likelihood).
Walter Sosa-Escudero ML Based Hypothesis Testing
Basic ConceptsThe Trilogy
Application: Test for Heteroskedasticity
Setup:yi = x′iβ + ui
1 xi is a non-stochastic vector of K explanatory variables,including an intercept.
2 ui ∼ N(0, σ2i ) (possible heteroskedasticity)
3 σ2i = h(α1 + α2z2i + α3z3i + . . .+ αpzpi).
4 h(.) is any positive, twice differentiable function.
5 zi is a vector of p non-random variables.
Homoskedasticity: H0 : α2 = α3 = · · · = αp = 0
Walter Sosa-Escudero ML Based Hypothesis Testing
Basic ConceptsThe Trilogy
Application: Test for Heteroskedasticity
Preliminary results I: LM tests for subvectors
In this setupθ = (β, α1, α2, . . . , αp)′.
The null of homskedasticity, H0 : α2, . . . , αp = 0 involves asubvector of θ, say, all the other parameters are free.
The LM test can be simplified in this situation
Walter Sosa-Escudero ML Based Hypothesis Testing
Basic ConceptsThe Trilogy
Application: Test for Heteroskedasticity
Remember that the null h(θ) = 0 can be tested based on:
LM = n s(θR)′ V −1n s(θR)
where Vn is consistent for J and θR is the restricted MLE.Consider now the case:
θ = (θ1 θ2)′, H0 : θ2 = θ20, θ2 is r × 1.
Note that:
H0 :[
0 Ir] [ θ1
θ2
]− θ2o︸ ︷︷ ︸
h(θ)
= 0
with D(θ) = [0 Ir].
Walter Sosa-Escudero ML Based Hypothesis Testing
Basic ConceptsThe Trilogy
Application: Test for Heteroskedasticity
Similarly
s(θ)[s1(θ)s1(θ)
], Vn(θ) =
[V11 V12
V21 V22
], V −1
n (θ) ≡[V 11 V 12
V 21 V 22
]
Walter Sosa-Escudero ML Based Hypothesis Testing
Basic ConceptsThe Trilogy
Application: Test for Heteroskedasticity
Result (Theil, 1971, pp. 18): V 22 =[V22 − V21V
−111 V12
]−1.
In particular, if V21 = 0 =⇒ V 22 = V −122 .
FOC of the restricted MLE problem: in this case, θR = (θ1, θ2o) soas:
s1(θ1, θ2o) = 0s2(θ1, θ2o) = λ
θ2 = θ2o
Walter Sosa-Escudero ML Based Hypothesis Testing
Basic ConceptsThe Trilogy
Application: Test for Heteroskedasticity
Reeplacing and simplifying in the LM formula:
LM = n s2(θ1; θ2o)′ V 22(θ) s2(θ1; θ2o)
If V21 = 0
LM = n s2(θ1; θ2o)′ V22(θ) s2(θ1; θ2o)
Walter Sosa-Escudero ML Based Hypothesis Testing
Basic ConceptsThe Trilogy
Application: Test for Heteroskedasticity
Preliminary results II: some OLS algebra
Y = Xβ + u, Xn×K including intercept.
ESS =∑
(Yi− Y )2 =∑
Y 2i −NY 2 =
∑Y 2i − (1/N)(
∑Yi)2
In matrix form:
SCE = Y ′Y − (i′Y )2/N, con i = (1, 1, . . . , 1)′
But:
Y ′Y = Y ′X(X ′X)−1X ′︸ ︷︷ ︸(Xβ′)
X(X ′X)−1X ′Y︸ ︷︷ ︸Xβ
= Y ′X(X ′X)−1X ′Y
replacing: SCE = Y ′X(X ′X)−1X ′Y − (i′Y )2/N
Walter Sosa-Escudero ML Based Hypothesis Testing
Basic ConceptsThe Trilogy
Application: Test for Heteroskedasticity
The Breusch-Pagan Test
Let
ei, OLS residuals.
σ2 = (1/N)∑e2i , MLE of σ2 under H0.
gt ≡ e2i /σ
2.
Test (Breusch-Pagan): Under H0 :
LM =12SCEg,z ∼ χ2(p− 1)
ESSg,z = ESS of regressing gi on zi.
Walter Sosa-Escudero ML Based Hypothesis Testing
Basic ConceptsThe Trilogy
Application: Test for Heteroskedasticity
Proof: yi ∼ N(x′iβ, σ2i ), σ
2i = h(z′iα)
l(β, α) = −12N ln(2π)− 1
2
∑lnσ2
i −12
∑(1/σ2
i )(yi − x′iβ
)2Note
s(β; x, z) =∑ 1
σ2i
(yi − x′iβ)xi
and
s(α; x, z) = −12
∑ h′ihizi −
12
∑ h′ih2i
zi
with hi ≡ h(z′iα), h′i ≡ ∂h(z′iα)/∂α.
Walter Sosa-Escudero ML Based Hypothesis Testing
Basic ConceptsThe Trilogy
Application: Test for Heteroskedasticity
Let β0 and α0 denote the true values under the null. Also notethat α0 = (α1, 0, . . . , 0)′.
From the previous result, it is easy to check that J is blockdiagonal, that is
Jα,β = −E[∂2l
∂α∂β
]= 0
when evaluated at the true values under the null. Then, since our
H0 only involves the components of α, according to our previousresult, a test can be based on
LM =1nsα(θR; z, x)′J−1
αα sα(θR; z, x)
Walter Sosa-Escudero ML Based Hypothesis Testing
Basic ConceptsThe Trilogy
Application: Test for Heteroskedasticity
Let β denote the MLE of β under the null. What it β in this case?.
Under H0 of homoskedasticity, let σ2 denote the MLE ofV (ui) = h(α1). By the invariance property, the MLE of α1 (α1) isdefined implicitely by σh(α1).
It is easy to verify
sα(θR) =12
[h′(α1)σ2
]∑zi
(e2i
σ2− 1)
and
Jαα =∂l(θ; x, z)∂αα′
∣∣∣∣θ=θR
=12
[h′(α1)σ2
]∑ziz′i
Walter Sosa-Escudero ML Based Hypothesis Testing
Basic ConceptsThe Trilogy
Application: Test for Heteroskedasticity
Replacing and simplifying:
LM =12
(∑zifi
)′ (∑ziz′i
)−1 (∑zifi
)with fi ≡ e2
i /σ2 − 1 = gi − 1. In matrix terms:
LM =12f ′Z(Z ′Z)−1Z ′f =
12f ′PZf
with f = g − i, i is a vector of n ones.
Walter Sosa-Escudero ML Based Hypothesis Testing
Basic ConceptsThe Trilogy
Application: Test for Heteroskedasticity
Note
i′g =∑e2i /σ
2 = n.
PZi = i (Why?).
Then
LM = 1/2 f ′PZf= 1/2 (g − i)′PZ(g − i)= 1/2
[g′PZg − i′PZg − g′PZi+ i′PZi
]= 1/2
[g′PZg − n
]= 1/2
[g′PZg − (i′g)2/n
]= 1/2 ESSg,z
Walter Sosa-Escudero ML Based Hypothesis Testing
Basic ConceptsThe Trilogy
Application: Test for Heteroskedasticity
On the small sample performance
Key result: under all the assumptions andH0 : LMbp ∼ χ2(p− 1), asymptotically.
This result is used to determine the acceptance region, for anexogenously set level α.
How reliable is this this result?
A key assumption is normality.
Walter Sosa-Escudero ML Based Hypothesis Testing
Basic ConceptsThe Trilogy
Application: Test for Heteroskedasticity
Evans (1992): a Monte Carlo exploration
Setup:
yi = x′iβ + ui
xi is generated using a log-normal and a uniform distribution
V (ui) = σ2(1 + λzi).
2000 replications for alternative disributions for ui.
n = 64, α = 0.05.
Tests: Breusch-Pagan (BPTrue), Modified BP (BPasym),Koenker (BPmod), White.
Walter Sosa-Escudero ML Based Hypothesis Testing