1 Hypothesis Testing Under General Linear Model Previously we derived the sampling property results...

1

Hypothesis Testing UnderGeneral Linear Model

Previously we derived the sampling property results assuming normality:Y = X + e where et~N(0,2)→ Y~N(X,2IT)s=(X'X)-1X'Y, E(s)=Cov(s)= β =2(X'X)-1

l~N(, 2(X'X)-1)σU

2 unbiased estimate of σ2

An estimate of Cov(βs) = βs=σU

2(X'X)-1

2 l lU

e eσ =

(T-K)

el = y - Xβl

2


Single Parameter (βk,L) Hypothesis Test βk,l~N(βk,Var(βk))

kth diagonal element of βs

When σ2 is known:

( )

, ~ (0,1)var

k l k

k

z Nb b

b

-=

unknown true coeff.

When σ2 not known:

( )

, ~ˆvar

k l kT K

k

t tb b

b-

-=

Σβs=σu2(X'X)-1

3


Can obtain (1-) CI for βk:

There is a (1-α) probability that the true unknown value of β is within this range

Does this interval contain our hypothesized value? If it does, than we can not reject H0

( )

( )

k,l k α 2,T-K k

k,l k α 2,T-K

ˆβ - var β t β

ˆβ + var β t

£

£

4


Testing More Than One Linear Combination of Estimated Coefficients Assume we have a-priori

information about the value of β

We can represent this information via a set of J-Linear hypotheses (or restrictions):

In matrix notation

K

jk k jk=1

R β = r j=1,2,…,J (J K)

(JxK) (Jx1)(Kx1)R β = r

5


(JxK) (Jx1)(Kx1)R β = r

11 12 1

21 22 2

1 2

K

k

J J JK

R R R

R R RR

R R R

1

2

J

r

rr

r

knowncoefficients

6


Assume we have a model with 5 parameters to be estimatedJoint hypotheses: β1=8 and β2=β3

J=2, K=5

0

1

2

3

4

0 1 0 0 0 8

0 0 1 1 0 0

0 1 0 0 0 8

0 0 1 1 0 0

R r

R r

β2-β3=0

7


How do we obtain parameter estimates if J hypotheses are true? Constrained (Restricted) Least

Squares R is β that minimizes: S=(Y-Xβ)'(Y-Xβ) s.t. Rβ=r

= e'e s.t. Rβ=re.g. we act as if H0 are true

S*=(Y-Xβ)'(Y-Xβ)+λ'(r-Rβ) λ is (J x1) Lagrangian multipliers

associated with J-joint hypotheses We want to choose β such that we

minimize SSE but also satisfy the J constraints (hypotheses), βR

8


Min. S*=(Y-Xβ)'(Y-Xβ) + λ'(r-Rβ)

What and how many FOC’s? K+J FOC’s

1

* * *0

K

S S S

K-FOC’s

*0

Sr R

J-FOC’s

9


What are the FOC’s?

( 1)

-1 -1R

*-2X Y + 2X Xβ - R λ 0

move -2X Y and -R λ to RHS

and divide by 2

R λX Xβ = X Y +

2R λ

β = X X X Y + X X2

Kx

S

Substitute these FOC into 2nd set∂S*/∂λ = (r-RβR) = 0J →

-1sR λ

r = Rβ + R X X2

S*=(Y-Xβ)'(Y-Xβ)+λ'(r-Rβ)

CRM

βS

10


-1s

-1-1s

λr-Rβ = R X X R 2

λ = R X X R r-Rβ2

The 1st FOC

Substitute the expression for λ/2 into the 1st FOC:

R s

1-1 -1sX X R R X X R r-Rβ

1 1 RX X X Y X X

2R

11


βR is the restricted LS estimator of β as well as the restricted ML estimator

Properties of Restricted Least Squares Estimator

→E(R) if R rV(R) ≤ V(S)

→[V(S) - V(R)] is positive semi-definite

diag(V(R)) ≤ diag(V(S))

R

1-1 -1

E

X X R R X X R r-Rβ

True butUnknown Value

12


From above, if Y is multivariate normal and H0 is true βl,R~N(β,σ2M*(X'X)-1M*')

~N(β,σ2M*(X'X)-1)

From previous results, if r-Rβ≠0 (e.g., not all H0 true), estimate of β is biased if we continue to assume r-Rβ=0

,

11 1

Bias l RE

X X R R X X R r R

≠0

11 1*

KM I X X R R X X RR

13


The variance is the same regardless of he correctness of the restrictions and the biasedness of βR → βR has a variance that is smaller

when compared to βs which only uses the sample information.

14


Beer Consumption Example : qB ≡ quantity of beer purchased

PB ≡ price of beerPL ≡ price of other alcoholic bev.PO≡ price of other goodsINC ≡ household income

Real Prices Matter? All prices and INC by 10% β1 + β2 + β3 + β4=0

Equal Price Impacts? Liquor and Other Goods β2=β3

Unitary Income Elasticity? β4=1

Data used in the analysis

ββ β β31 2 4B B L Oq =αP P P INC exp( )e

15

Given the above, what does the R-matrix and r vector look like for these joint tests?

Lets develop a test statistic to test these joint hypotheses

We are going to use the Likelihood Ratio (LR) to test the joint hypotheses


0 1 1 1 1 0

0 0 1 1 0 0

0 0 0 0 1 1

R r

B,t 0 1 B,t 2 L,t

3 O,t 4 t t

lnq =β +β ln(P )+β ln(P )

+β ln(P )+β ln(Inc )+e

16


LR=lU*/lR

*

lU*=Max [l(|y1,…,yT);

=(β, σ) ]= “unrestricted” maximum likelihood function

lR*=Max [l(|y1,…,yT);

=(β, σ); Rβ=r]= “restricted” maximum likelihood function

Again, because we are possibly restricting the parameter space via our null hypotheses, LR≥1

17


If lU* is large relative to lR

*→data shows evidence that the restrictions (hypotheses) are not true (e.g., reject null hypothesis) How much should LR exceed 1

before we reject H0? We reject H0 when LR ≥ LRC where

LRC is a constant chosen on the basis of the relative cost of the Type I vs. Type II errors

When implementing the LR Test you need to know the PDF of the dependent variable which determines the density of the test statistic

18


For LR test, assume Y has a normal distribution →e~N(0,σIT) This implies the following LR

test statistic (LR*) What are the distributional

characteristics of LR*? Will address this in a bit

19


We can derive alternative specifications of LR test statistic LR*=(SSER-SSEU)/(J2U)

(ver. 1) LR*=[(Re-r)′[R(X′X)-1R′]-1(Re-r)]/(J2U)

(ver. 2) LR*=[(R-e)′(X′X)(R-e)]/(J2U)

(ver. 3)βe =βS=βl

What are the Distributional Characteristics of LR* (JHGLL p. 255) LR* ~ FJ,T-K

J = # of Hypotheses K= # of Parameters (including intercept)

20


Proposed Test Procedure Choose = P(reject H0| H0 true) =

P(Type-I error) Calculate the test statistic LR*

based on sample information Find the critical value LRcrit in an F-

table such that: = P(F(J, T – K) LRcrit), where α =

P(reject H0| H0 true)f(LR*)

αLRcrit

α = P(FJ,T-K ≥ LRcrit)

21


Proposed Test Procedure Choose = P(reject H0| H0 true) =

P(Type-I error) Calculate the test statistic LR*

based on sample information Find the critical value LRcrit in an F-

table such that: = P(F(J, T – K) LRcrit), where α =

P(reject H0| H0 true) Reject H0 if LR* LRcrit

Don’t reject H0 if LR* < LRcrit

22


Beer Consumption Example

Does the regression do a better job in explaining variation in beer consumption than if assumed the mean response across all obs.? Remember SSE=(T-K)σ2

U

Under H0: All slope coefficients=0

Under H0, TSS=SSE given that that there is no RSS and TSS=RSS+SSE

B,t 0 1 B,t 2 L,t

3 O,t 4 t t



23


Log-Log Beer Consumption Model

Unconstrained Model

R2 0.8254

Adj. R2 0.7975

σU 0.05997

Obs 30

Variable Coeff Std Error T-Stat

Intercept -3.243 3.743 -0.87

lnPB -1.020 0.239 -4.27

lnPL -0.583 0.560 -1.04

lnPO 0.210 0.080 2.63

ln(INC) 0.923 0.416 2.22

Constrained Model

σU 0.13326 SSER=0.133262*29=0.51497

Coeff Std Error T-Stat

Intercept 4.019 0.0243 165.17

SSE = 0.059972 *25 = 0.08992

R2=1- 0.08992/0.51497

TSS=SSERMean of LN(Beer)

24


Results of our test of overall significance of regression model

Lets look at the following GAUSS Code

GAUSS command:CDFFC(29.544,4,25)=3.799e-009CDFFC Computes the complement

of the cdf of the F distribution (1-Fdf1,df2)

Unlikely value of F if hypothesis is true, that is no impact of exogenous variables on beer consumption

Reject the null hypothesisAn alternative look

25


Beer Consumption Example

Three joint hypotheses exampleSum of Price and Income

Elasticities Sum to 0 (e.g., β1 + β2 + β3 + β4=0)Other Liquor and Other Goods

Price Elasticities are Equal (e.g., β2=β3)

Income Elasticity = 1 (e.g., β4=1) cdffc(0.84,3,25)=0.4848

B,t 0 1 B,t 2 L,t

3 O,t 4 t t



26


PDF

F3,25

0.84

area = 0.4848

Location of our calculated test statistic

F

27


A side note: How do you estimate the variance of an elasticity and therefore test H0 about this elasticity?

Suppose you have the following model:FDXt = β0 + β1Inct + β2 Inc2

t + et

FDX= food expenditure Inc=household income

Want to estimate the impacts of a change in income on expenditures. Use an elasticity measure evaluated at mean of the data. That is:

28


Income Elasticity (Γ) is:

How do you calculate the variance of Γ?

We know that: Var(α′Z)= α′Var(Z)α Z is a column vector of RV’s α a column vector of constants

Treat β0, β1 and β2 are RV’s. The α vector is:

1 2FDX Inc Inc

2 IncInc FDX FDX

2Inc 2Inc

α 0FDX FDX

FDXt = β0 + β1Inct + β2 Inc2t + et

Linear combination of Z

29


This implies var(Γ) is:

2 0 0 1 0 2

0 1 1 1 2

0 2 1 2 2 2

0Var β Cov β ,β Cov β ,β

Inc 2Inc Inc0 Cov β ,β Var β Cov β ,β

FDX FDX FDXCov β ,β Cov β ,β Var β

2Inc

FDX

21 1 2

21 2 2

(2 2)(1 2)

(2 1)

Inc

Var β Cov β ,β FDXInc 2Inc

Cov β ,β Var βFDX FDX 2Inc

FDXxx

x

(1 x 1)

σ2(X'X)-1

(3 x 3)

(1 x 3)

(3 x 1)Due to 0 α value

30


This implies: var(Γ) is:

22 2

1 2

2

1 2

Inc 2IncVar Var β Var β

FDX FDX

Inc 2Inc2 Cov β ,β

FDX FDX

C12

C22

22 2

1 2

3

1 22

Inc 2IncVar β Var β

FDX FDX

Inc4 Cov β ,β

FDX

2C1C2

1 Hypothesis Testing Under General Linear Model Previously we derived the sampling property results...

Documents

Transcript of 1 Hypothesis Testing Under General Linear Model Previously we derived the sampling property results...