In linear regression model Yi = i2 u , i n - Personal World Wide...

26
Lecture 17. Estimation with heteroskedastic errors; serial correlation In linear regression model i iK K i i i u X X X Y + + + + = β β β " 2 2 1 1 , n i , , 1 = that random error term is heteroskedastic if 2 2 ) ( ) ( i i i u E u Var σ = = , n i , , 1 = i.e. if the variance of the random error is different for each observation.

Transcript of In linear regression model Yi = i2 u , i n - Personal World Wide...

Lecture 17. Estimation with heteroskedastic errors; serial correlation In linear regression model

iiKKiii uXXXY ++++= βββ 2211 , ni ,,1…= that random error term is heteroskedastic if 22 )()( iii uEuVar σ== , ni ,,1…= i.e. if the variance of the random error is different for each observation.

Summary of previous lecture

• Heteroskedastic errors if there are omitted variables that have a different order of magnitude for the observations, e.g. cross-section data on US states

• OLS estimator of the regression coefficient is still unbiased

• OLS estimator is not the best estimator • Usual formula for standard errors and t-

and F-tests are not correct, i.e. the computer output that uses these formulas cannot be used

• You can find out whether the errors are heteroskedastic by plotting the square of the OLS residuals against variables

• Better is to use the LM test which depends on a particular model for the heteroskedasticity, e.g. BP model. Then the test statistic is the number of observations times the 2R of the regression of the squared OLS residuals on a number of variables that are suspected to affect that variance of the random error term

• There is an alternative method to compute the standard errors of the OLS estimates that is valid even if the random errors are heteroskedastic

Improving on OLS Consider the (simple) linear regression model with heteroskedastic errors

iii uXY ++= 221 ββ

2)( iiuVar σ= How can we transform this into a linear regression model with a homoskedastic random error term? Remember for a constant c 22)( ii ccuVar σ= i.e. if we multiply a variable by a constant the variance is multiplied by the square of that constant.

Hence if we choose i

cσ1

= we have

1=

I

iuVar

σ

and the regression model

(1) i

i

i

i

ii

i uXY

σσβ

σβ

σ++= 2

211

with dependent variable i

iY

σ and independent

variables iσ

1 and i

iX

σ2 has a homoskedastic

random error term (and the same regression coefficients 1β and 2β . If we estimate the regression coefficients in this model by OLS we again get the BLU estimators of the regression coefficients. Note this model has no constant term.

The OLS estimators of 1β and 2β minimize

( )∑∑==

−−=

−−

n

iii

i

n

i i

i

ii

i XYXY

1

22212

2

1

221

11ββ

σσβ

σβ

σ

The last expression is a weighted sum of squared residuals with weights equal to 1 over the variance of the random error. The OLS estimators in model (1) are called Weighted Least Squares (WLS) estimators. This is a special case of a Generalized Least Squares (GLS) estimator. GLS estimators are the best estimators if the assumptions 3 or 4 in the CLR model do not hold. Problem with WLS estimator: In general 2

iσ is not known.

Special case in which we can use WLS estimator directly: Error variance proportional with square of size variable iZ 22)( ii ZuVar σ= Example: data are cross-section data on US states and iZ is the size of the population of state i . Now

2σ=

i

i

Z

uVar

and if we divide the dependent and independent variables (including the constant!) by iZ we obtain a linear regression model with homoskedastic errors. The resulting OLS estimator is the WLS estimator and BLU.

In general case we start with a model of the variance of the random error (as we did in deriving the LM test), e.g. the HG model

iLLii ZZ ααασ +++= L2212log (Harvey-

Godfrey) The first 2 steps are as in the LM test

1. Estimate by OLS and obtain OLS residuals niei ,,1, K=

2. Estimate linear regression of 2log ie on constant and iLi ZZ ,,2 K and compute

)ˆˆˆexp(ˆ 2212

iLLii ZZ ααασ +++= L 3. Divide the dependent and independent

variables by iσ̂ and estimate the regression coefficients by OLS

This estimator is called the Feasible GLS or Feasible WLS estimator.

Surprising fact: The standard errors of the FGLS estimators obtained in step 3 are correct as are the t- and F-tests in this model (strictly this is true if the sample is large). The fact that we use estimated variances 2ˆ iσ does not matter!

Application to relation between log salary and experience (in years since Ph.D.) Data on 222 university professors for 7 schools (UC Berkeley, UCLA, UCSD, Illinois, Stanford, Michigan, Virginia) Model for error variance 2

3212log YearsYearsi ααασ ++=

• Estimates of 321 ,, ααα • Compare OLS estimates and FGLS

estimates • Compare OLS standard error,

heteroskedasticity-consistent standard error, FGLS standard error

• FGLS standard error is smallest

Dependent Variable: LNRESID2Method: Least SquaresDate: 11/12/01 Time: 23:16Sample: 1 222Included observations: 222

Variable Coefficient Std. Error t-Statistic Prob.

C -6.562664 0.359239 -18.26824 0.0000YEARS 0.235562 0.041963 5.613595 0.0000

YEARS2 -0.004776 0.001050 -4.547694 0.0000

R-squared 0.159023 Mean dependent var -4.404772Adjusted R-squared 0.151342 S.D. dependent var 1.952342S.E. of regression 1.798549 Akaike info criterion 4.025259Sum squared resid 708.4163 Schwarz criterion 4.071241Log likelihood -443.8037 F-statistic 20.70563Durbin-Watson stat 1.565414 Prob(F-statistic) 0.000000

Dependent Variable: LNSALARYMethod: Least SquaresDate: 11/07/01 Time: 13:15Sample: 1 222Included observations: 222

Variable Coefficient Std. Error t-Statistic Prob.

C 3.809365 0.041338 92.15104 0.0000YEARS 0.043853 0.004829 9.081645 0.0000

YEARS2 -0.000627 0.000121 -5.190657 0.0000

R-squared 0.536179 Mean dependent var 4.325410Adjusted R-squared 0.531943 S.D. dependent var 0.302511S.E. of regression 0.206962 Akaike info criterion -0.299140Sum squared resid 9.380504 Schwarz criterion -0.253158Log likelihood 36.20452 F-statistic 126.5823Durbin-Watson stat 1.434005 Prob(F-statistic) 0.000000

Dependent Variable: LNSALARYMethod: Least SquaresDate: 11/07/01 Time: 13:49Sample: 1 222Included observations: 222White Heteroskedasticity-Consistent Standard Errors & Covariance

Variable Coefficient Std. Error t-Statistic Prob.

C 3.809365 0.026119 145.8466 0.0000YEARS 0.043853 0.004361 10.05599 0.0000

YEARS2 -0.000627 0.000118 -5.322369 0.0000

R-squared 0.536179 Mean dependent var 4.325410Adjusted R-squared 0.531943 S.D. dependent var 0.302511S.E. of regression 0.206962 Akaike info criterion -0.299140Sum squared resid 9.380504 Schwarz criterion -0.253158Log likelihood 36.20452 F-statistic 126.5823Durbin-Watson stat 1.434005 Prob(F-statistic) 0.000000

Dependent Variable: LNSALARYWLSMethod: Least SquaresDate: 11/12/01 Time: 23:24Sample: 1 222Included observations: 222

Variable Coefficient Std. Error t-Statistic Prob.

CWLS 3.827501 0.020303 188.5145 0.0000YEARSWLS 0.038216 0.003257 11.73423 0.0000

YEARS2WLS -0.000443 8.27E-05 -5.359798 0.0000

R-squared 0.989833 Mean dependent var 41.66265Adjusted R-squared 0.989740 S.D. dependent var 16.59118S.E. of regression 1.680524 Akaike info criterion 3.889510Sum squared resid 618.4915 Schwarz criterion 3.935492Log likelihood -428.7356 Durbin-Watson stat 1.324687

Model estimated in step 3 seems to fit better ( 2R is larger). However dependent variables in OLS and transformed OLS model are different: lnSalary and lnSalary divided by

iσ̂ , respectively. To find 2R for WLS we compute the WLS residuals 2

321ˆˆˆlog iiii YearsYearsSalarye βββ −−−=

where we use the FGLS estimators. These residuals are used in the usual formula for 2R with dependent variable Salarylog .

Serial Correlation What we will discuss

• Nature of economic time series • Consequences of this for random error

term in regression model • Model for serial correlation of random

error term • Consequences for OLS estimator of

regression coefficients • Detecting serial correlation

Most economic time series change gradually over time Example: US GNP (billions of 1982$) and new housing units (thousands) for 1963-1985 (see time series graph)

1000

1500

2000

2500

3000

3500

4000

64 66 68 70 72 74 76 78 80 82 84

GNP HOUSING

A numerical measure for the slowness of change or persistence of the time series is the autocorrelation coefficient. For time series

ntYt ,,1, K= the autocorrelation coefficient of order 1 is defined as the sample correlation between tY and 1−tY . The time series ntYt ,,2,1 K=− is the time series

tY lagged by one period. The value of 1−tY in period t is the value of Y in period 1−t . If 0Y is not known, the lagged once time series starts in period 2=t .

Define the sample average

∑=

=n

ttY

nY

1

1

then the autocorrelation coefficient of order 1 is

(2)

=

=−

−−=

n

tt

n

ttt

YY

YYYY

1

2

21

1

)(

))((

ρ̂

Note that by the definition of the sample correlation the denominator should be

∑∑=

−=

−−n

tt

n

tt YYYY

2

21

1

2 )()(

The only difference is with the way the first observation is included and that difference can be neglected. (2) is simpler.

The sample correlation between tY and 2−tY is the autcorrelation coefficient of order 2, 2ρ̂ etc. The autocorrelation coefficient of order k is a measure of the (linear) relation between the time series tY and ktY − . What do you expect for the values K,ˆ,ˆ 21 ρρ ? Why? Example: autocorrelation for GNP series.

Correlogram of GNP

Date: 11/13/01 Time: 06:27Sample: 1963 1985Included observations: 23

Autocorrelation Partial Correlation AC PAC Q-Stat Prob

1 0.840 0.840 18.426 0.0002 0.684 -0.071 31.243 0.0003 0.568 0.042 40.521 0.0004 0.479 0.018 47.454 0.0005 0.372 -0.107 51.882 0.0006 0.271 -0.044 54.358 0.0007 0.157 -0.125 55.244 0.0008 0.032 -0.140 55.284 0.000

Consequence for linear regression model In linear regression model ttt uXY ++= 231 ββ nt ,,1 K= the error term tu captures the omitted variables that affect Y . These variables are also economic time series and can have the same persistence as GNP. How do we express that tu and 1−tu are related? Suggestion use a linear regression model (3) ttt uu ερ += −1 Note: no constant because 0)()( 1 == −tt uEuE

The error term tε has the same properties as the error term in the CLR model, in particular 0)( =tE ε

2)( σε =tVar (homoskedasticity) for 0≠s , tε and st−ε are uncorrelated

Such a time series is called a white noise series (if fed to your speakers you will hear static) The model in (3) with tε white noise is called the first-order autoregressive or AR(1) process. The parameter ρ is called the first-order autocorrelation coefficient.

It can be shown that (3) implies that the correlation between tu and stu − is equal to

sρ Remember that in economic time series the correlation becomes smaller with s . This happens if 11 <<− ρ .

Consequences of serial correlation

• Assumptions 1 and 2 still hold: OLS estimators unbiased

• OlS estimator not the best (BLU) estimator

• Usual formula for standard error incorrect

• t- and F-tests cannot be used Compare with consequences of heteroskedasticity Often: standard errors produced by computer OLS program too small

Detecting serial/autocorrelation

• Graphical method • Test

Graphical method Data for 1963-1985. Dependent variable log of housing units started per capita and independent variables log of GNP per capita and log of mortgage interest rate See regression output To detect serial correlation we look at the OLS residuals

• Time series graph • Scatterplot of te and 1−te

Problem: indicative but not conclusive.

Dependent Variable: LNHOUSINGCAPMethod: Least SquaresDate: 11/13/01 Time: 00:06Sample: 1963 1985Included observations: 23

Variable Coefficient Std. Error t-Statistic Prob.

C 2.528899 1.180472 2.142278 0.0447LNGNPCAP -0.066000 0.540505 -0.122109 0.9040LNINTRATE -0.211284 0.202894 -1.041351 0.3101

R-squared 0.094147 Mean dependent var 1.991961Adjusted R-squared 0.003562 S.D. dependent var 0.226095S.E. of regression 0.225692 Akaike info criterion -0.018186Sum squared resid 1.018735 Schwarz criterion 0.129922Log likelihood 3.209133 F-statistic 1.039325Durbin-Watson stat 0.913015 Prob(F-statistic) 0.372027

-0.4

-0.2

0.0

0.2

0.4

0.6

64 66 68 70 72 74 76 78 80 82 84

LNHOUSINGCAP Residuals

-0.4

-0.2

0.0

0.2

0.4

0.6

-0.4 -0.2 0.0 0.2 0.4 0.6

RESID01

RE

SID

01LA

G

RESID01LAG vs. RESID01