Finite Sample Properties of Finite Sample Properties of S2

Finite-Sample Properties of OLS in the ClassicalLinear Model

Walter Sosa-Escudero

Econ 507. Econometric Analysis. Spring 2009

January 21, 2009

Walter Sosa-Escudero Finite-Sample Properties of OLS in the Classical Linear Model

Finite Sample Properties of Finite Sample Properties of S2

Classical linear model:

1 Linearity: Y = X + u.2 Strict exogeneity: E(u|X) = 03 No Multicollinearity: (X) = K.4 No heteroskedasticity/ serial correlation: V (u|X) = 2X.

OLS Estimator: = (X X)1X YEstimator of 2: S2 =

2i=1 e

2i /(nK).

Finite sample properties: properties of and S2 that can beverified for any fixed sample size n.

The goal is to derive some basic properties that hold under theclassical linear model.

Walter Sosa-Escudero Finite-Sample Properties of OLS in the Classical Linear Model

Finite Sample Properties of Finite Sample Properties of S2

Finite Sample Properties of

Properties of

1 Unbiasedness: E(|X) = .2 Variance expression: V (|X) = 2(X X)1.3 Gauss/Markov Theorem: is the best linear unbiased

estimator.

Walter Sosa-Escudero Finite-Sample Properties of OLS in the Classical Linear Model

Finite Sample Properties of Finite Sample Properties of S2

Unbiasedness: E(|X) =

= (X X)1X Y= (X X)1X (X + u)= + (X X)1X u

E(|X) = + E[(X X)1X u|X

]= + (X X)1X E [u|X]= (Since E(u|X) = 0)

How does heteroskedasticity affect unbiasedness?Which assumptions do we use and which ones we dont?

Walter Sosa-Escudero Finite-Sample Properties of OLS in the Classical Linear Model

Finite Sample Properties of Finite Sample Properties of S2

Can we prove unconditional unbiasedness: E() = ?

Using the LIE:

E() = E[E(|X)

]= E() =

Walter Sosa-Escudero Finite-Sample Properties of OLS in the Classical Linear Model

Finite Sample Properties of Finite Sample Properties of S2

Variance expression: V (|X) = 2(X X)1.

V (|X) = V ( |X) ( is not-random)= V

[(X X)1X u|X

](from previous proof...)

= E[(X X)1X uuX(X X)1 |X

]= (X X)1X E(uu|X)X(X X)1

= (X X)1X 2InX(X X)1 (by Assumption 4)= 2(X X)1

Walter Sosa-Escudero Finite-Sample Properties of OLS in the Classical Linear Model

Finite Sample Properties of Finite Sample Properties of S2

Can we derive an expression for the unconditional variance, V ().?Sure, but we need an extra result

Result: V () = E[V (|X)] + V[E(|X)

]By unbiasedness, E(|X) = , so V () = 0. Also, by our previousresult V (|X) = 2(X X)1. Replacing above:

V () = E[2(X X)1

]= 2E

[(X X)1

]

Walter Sosa-Escudero Finite-Sample Properties of OLS in the Classical Linear Model

Finite Sample Properties of Finite Sample Properties of S2

Gauss/Markow Theorem: is the best linear unbieased estimator.

Formally: For the classical linear model, for any linear unbiasedestimator ,

V (|X) V (|X) 0

that is, V (|X) V (|X) is a positive semidefinite matrix.

Before attacking the proof: better stands for smaller variance.So the GMT says that among all linear and unbiased estimators of for the classical linear model, is the best

It is a rather restrictive notion.

Walter Sosa-Escudero Finite-Sample Properties of OLS in the Classical Linear Model

Finite Sample Properties of Finite Sample Properties of S2

Proof:

linear: there is AKn that depends on X, with rank K, suchthat = AY .Under the classical linear model

E(|X) = E(AY |X) = E (A(X + u)|X) = AX (1)

unbiased:E(|X) = (2)

linear and unbiased: (1) and (2) hold simultaneouly. Thisrequires AX = I.

Walter Sosa-Escudero Finite-Sample Properties of OLS in the Classical Linear Model

Finite Sample Properties of Finite Sample Properties of S2

Trivially, = + + , with .

Note that: V (|X) = V (|X) + V (|X) iff Cov(, |X) = 0.

So if we prove Cov(, |X) = 0, we have the result. Why?

Walter Sosa-Escudero Finite-Sample Properties of OLS in the Classical Linear Model

Finite Sample Properties of Finite Sample Properties of S2

First note that trivially E(|X) = 0 (Why?)

Hence: Cov(, | X) = E[( ) | X]

Note that

= AY (X X)1X Y= (A (X X)1X )Y= (A (X X)1X )(X + u)= (A (X X)1X )u (since AX = I)

Walter Sosa-Escudero Finite-Sample Properties of OLS in the Classical Linear Model

Finite Sample Properties of Finite Sample Properties of S2

Now replace to get:

Cov(, | X) = E[( ) | X]= E[(X X)1X )uu(A (X X)1X ) | X]= 2[(X X)1X (A X(X X)1)]= 2[(X X)1X A (X X)1X X(X X)1)]= 0

where we used V (u|X) = E(uu|X) = 2In, and, again, AX = I.So, by our previous argument we get:

V ( | X) V ( | X) = V ( | X)

which is by construction positive semidefinite.

Walter Sosa-Escudero Finite-Sample Properties of OLS in the Classical Linear Model

Finite Sample Properties of Finite Sample Properties of S2

Can we obtain an unconditional version of the Gauss-MarkowTheorem? Yes!

Ill leave it as an exercise. See Problem 4b) (pp.32) in Hayashistext.

Walter Sosa-Escudero Finite-Sample Properties of OLS in the Classical Linear Model

Finite Sample Properties of Finite Sample Properties of S2

Finite Sample Properties of S2

Remember that we proposed:

S2 =e2i

nK=

ee

nK

as an estimator for 2.

Result: S2 is unbiased (E(S2|X) = 2)

Walter Sosa-Escudero Finite-Sample Properties of OLS in the Classical Linear Model

Finite Sample Properties of Finite Sample Properties of S2

Trace: Let A be a square mm matrix. Its trace of A is the sumof all its principal diagonal elements: tr(A) =

mi=1Aii.

Simple properties:

If A is a scalar, trivially tr(A) = Atr(AB) = tr(BA)tr(AB) = tr(A) + tr(B)

The M matrix: M In X(X X)1X

Some properties (check as homework)

M = M (symmetric), M = MM (idempotent)tr(M) = nKe = Mu.

e = Y X = Y X(XX)1XY = (I X(XX)1X)Y = MY = MX +Mu. Nownote that MX = (I X(XX)1X)X = X X(XX)1X)X = 0.

Walter Sosa-Escudero Finite-Sample Properties of OLS in the Classical Linear Model

Finite Sample Properties of Finite Sample Properties of S2

Proof:

E(S2 | X) = E(ee | X)

nK=

E(uM Mu | X)nK

=E(uMu | X)

nK

=E(tr(uMu) | X)

nK

=E(tr(uuM) | X)

nK

=tr(E(uuM | X))

nK

=tr(2InM)nK

=2tr(M)nK

= 2

Walter Sosa-Escudero Finite-Sample Properties of OLS in the Classical Linear Model

Finite Sample Properties of Finite Sample Properties of S2