Regression Models - University of Wisconsin–Madisonssc.wisc.edu/~bhansen/390/390Lecture18.pdf ·...

42
Regression Models Bivariate data (y,x) Multivariate (y,x 1 ,…,x k ) Suppose the conditional mean of y is a function of x Then the regression function is the optimal forecast of y given x ( ) t t t x x y E β α + = |

Transcript of Regression Models - University of Wisconsin–Madisonssc.wisc.edu/~bhansen/390/390Lecture18.pdf ·...

Page 1: Regression Models - University of Wisconsin–Madisonssc.wisc.edu/~bhansen/390/390Lecture18.pdf · 2014-04-01 · Regression Models • Bivariate data (y,x) • Multivariate (y,x

Regression Models

• Bivariate data (y,x)

• Multivariate (y,x1,…,xk)

• Suppose the conditional mean of  y is a function of x

• Then the regression function is the optimal forecast of  y  given  x

( ) ttt xxyE βα +=|

Page 2: Regression Models - University of Wisconsin–Madisonssc.wisc.edu/~bhansen/390/390Lecture18.pdf · 2014-04-01 · Regression Models • Bivariate data (y,x) • Multivariate (y,x

Regression

• Model

• Estimation: Least‐Squares

• Example: Interest Rates– Monthly

– Rates on 3‐month and 12‐month T‐Bills

ttt exy ++= βα

Page 3: Regression Models - University of Wisconsin–Madisonssc.wisc.edu/~bhansen/390/390Lecture18.pdf · 2014-04-01 · Regression Models • Bivariate data (y,x) • Multivariate (y,x

3‐month and 12‐month T‐Bill

Page 4: Regression Models - University of Wisconsin–Madisonssc.wisc.edu/~bhansen/390/390Lecture18.pdf · 2014-04-01 · Regression Models • Bivariate data (y,x) • Multivariate (y,x

Least‐Squares, 3‐month on 12‐month

Page 5: Regression Models - University of Wisconsin–Madisonssc.wisc.edu/~bhansen/390/390Lecture18.pdf · 2014-04-01 · Regression Models • Bivariate data (y,x) • Multivariate (y,x

Forecast

• A forecast of  yn+h requires xn+h• This is not typically feasible

hnhnhn exy +++ ++= βα

hnnhn xy ++ += βα|ˆ

Page 6: Regression Models - University of Wisconsin–Madisonssc.wisc.edu/~bhansen/390/390Lecture18.pdf · 2014-04-01 · Regression Models • Bivariate data (y,x) • Multivariate (y,x

Regressor forecast

• Suppose we have a forecast for  x

• Then

• For example, if

then

hnnhn xy ++ += ˆˆ | βα

( ) hthtt xxE −− +=Ω φγ|

nnhn xx φγ +=+ |ˆ

Page 7: Regression Models - University of Wisconsin–Madisonssc.wisc.edu/~bhansen/390/390Lecture18.pdf · 2014-04-01 · Regression Models • Bivariate data (y,x) • Multivariate (y,x

12‐month t‐bill on Lagged Value

• Regress  xt on  xt‐12  (12‐month ahead forecast)

Page 8: Regression Models - University of Wisconsin–Madisonssc.wisc.edu/~bhansen/390/390Lecture18.pdf · 2014-04-01 · Regression Models • Bivariate data (y,x) • Multivariate (y,x

3‐month on 12‐month

• Prediction using regression and fitted value

Page 9: Regression Models - University of Wisconsin–Madisonssc.wisc.edu/~bhansen/390/390Lecture18.pdf · 2014-04-01 · Regression Models • Bivariate data (y,x) • Multivariate (y,x

Interest Rate Forecast

• Estimates

• Current:  x2010M2= 0.35%

1285.084.0ˆ94.021.0ˆ

−+=+−=

tt

tt

xxxy

86.014.194.021.0ˆ14.135.085.084.0ˆ 22011

=×+−==×+=

t

M

yx

Page 10: Regression Models - University of Wisconsin–Madisonssc.wisc.edu/~bhansen/390/390Lecture18.pdf · 2014-04-01 · Regression Models • Bivariate data (y,x) • Multivariate (y,x

Example

• The AR(1) forecasts the 12‐month T‐bill next February to rise to 1.14%

• The regression model forecasts the 3‐month T‐bill next February to be 0.85% – Currently 0.11%

Page 11: Regression Models - University of Wisconsin–Madisonssc.wisc.edu/~bhansen/390/390Lecture18.pdf · 2014-04-01 · Regression Models • Bivariate data (y,x) • Multivariate (y,x

Direct Method (preferred)

• Combine

• We obtain

( ) ttt xxyE ϕα +=|

( ) hthtt xxE −− +=Ω φγ|

( ) ( )( )

ht

ht

htthtt

xx

xEyE

−−

+=++=

Ω+=Ω

βμφγϕα

ϕα ||

Page 12: Regression Models - University of Wisconsin–Madisonssc.wisc.edu/~bhansen/390/390Lecture18.pdf · 2014-04-01 · Regression Models • Bivariate data (y,x) • Multivariate (y,x

Forecast Regression

• h‐step‐ahead

• Forecast 

thtt exy ++= −βμ

nnhn xy βμ +=+ |

Page 13: Regression Models - University of Wisconsin–Madisonssc.wisc.edu/~bhansen/390/390Lecture18.pdf · 2014-04-01 · Regression Models • Bivariate data (y,x) • Multivariate (y,x

3‐month on Lagged 12‐month

nnhn xy 79.61.| +=+

89.035.079.61.ˆ | =⋅+=+ nhny

Page 14: Regression Models - University of Wisconsin–Madisonssc.wisc.edu/~bhansen/390/390Lecture18.pdf · 2014-04-01 · Regression Models • Bivariate data (y,x) • Multivariate (y,x

AR(q) Regressors

• Suppose  x  is an AR(q)

• Then a one‐step forecasting equation for  y is

• And an h‐step is

tqtqttt

ttt

uxxxxexy

+++++=++=

−−− φφφγβα

L2211

tqtqttt exxxy +++++= −−− βββμ L2211

tqhtqhthtt exxxy +++++= −+−−−− 1121 βββμ L

Page 15: Regression Models - University of Wisconsin–Madisonssc.wisc.edu/~bhansen/390/390Lecture18.pdf · 2014-04-01 · Regression Models • Bivariate data (y,x) • Multivariate (y,x

T‐Bill example: AR(12) for 12‐month

Page 16: Regression Models - University of Wisconsin–Madisonssc.wisc.edu/~bhansen/390/390Lecture18.pdf · 2014-04-01 · Regression Models • Bivariate data (y,x) • Multivariate (y,x

Regress 3‐month on 12 lags of 12‐month 

Page 17: Regression Models - University of Wisconsin–Madisonssc.wisc.edu/~bhansen/390/390Lecture18.pdf · 2014-04-01 · Regression Models • Bivariate data (y,x) • Multivariate (y,x

Forecast

• Predicted value for 2011M2=1.06

• Predicted value using 3 lags=0.91

Page 18: Regression Models - University of Wisconsin–Madisonssc.wisc.edu/~bhansen/390/390Lecture18.pdf · 2014-04-01 · Regression Models • Bivariate data (y,x) • Multivariate (y,x

Distributed Lags

• This class of models is called distributed lags

• If we interpret the coefficients as the effect of x on y, we sometimes say– β1 is the immediate impact

– β1+ …+ βn = B(1) is the long‐run impact

tt

tqtqttt

exLB

exxxy

++=

+++++=

−−−

1

2211

)(μ

βββμ L

Page 19: Regression Models - University of Wisconsin–Madisonssc.wisc.edu/~bhansen/390/390Lecture18.pdf · 2014-04-01 · Regression Models • Bivariate data (y,x) • Multivariate (y,x

Regressors and Dynamics

• We have seen AR forecasting models

• And now distributed lag model

• Add both together!

• orttt exLByLA ++= −1)()( μ

tqtqt

ptptt

exx

yyy

++++

+++=

−−

−−

ββ

ααμ

L

L

11

11

Page 20: Regression Models - University of Wisconsin–Madisonssc.wisc.edu/~bhansen/390/390Lecture18.pdf · 2014-04-01 · Regression Models • Bivariate data (y,x) • Multivariate (y,x

h‐step

• Regress on lags of  y  and  x,  h periods back

• Estimate by least squares

• Forecast using estimated coefficients and final values

tqhtqht

phtphtt

exx

yyy

++++

+++=

−+−−

−+−−

11

11

ββ

ααμ

L

L

Page 21: Regression Models - University of Wisconsin–Madisonssc.wisc.edu/~bhansen/390/390Lecture18.pdf · 2014-04-01 · Regression Models • Bivariate data (y,x) • Multivariate (y,x

3‐month t‐bill forecast

Page 22: Regression Models - University of Wisconsin–Madisonssc.wisc.edu/~bhansen/390/390Lecture18.pdf · 2014-04-01 · Regression Models • Bivariate data (y,x) • Multivariate (y,x

Forecast

• Predicted value for 2011M2=1.26

Page 23: Regression Models - University of Wisconsin–Madisonssc.wisc.edu/~bhansen/390/390Lecture18.pdf · 2014-04-01 · Regression Models • Bivariate data (y,x) • Multivariate (y,x

Model Selection

• The dynamic distributed lag model has p lags of  y and  q  lags of x, a total of 1+p+q estimated coefficients

• Models (p and q) can be selected by calculating and minimizing the AIC

• Penalty is 2 times number of estimated coefficients

( )12ln +++⎟⎠⎞

⎜⎝⎛= qp

TSSRTAIC

Page 24: Regression Models - University of Wisconsin–Madisonssc.wisc.edu/~bhansen/390/390Lecture18.pdf · 2014-04-01 · Regression Models • Bivariate data (y,x) • Multivariate (y,x

Predictive Causality

• The variable  x  affects a forecast for  y  if lagged values of  x  have true non‐zero coefficients in the dynamic regression of  y  on lagged y’s and lagged  x’s

• If one of the β’s are non‐zero

tqtqt

ptptt

exx

yyy

++++

+++=

−−

−−

ββ

ααμ

L

L

11

11

Page 25: Regression Models - University of Wisconsin–Madisonssc.wisc.edu/~bhansen/390/390Lecture18.pdf · 2014-04-01 · Regression Models • Bivariate data (y,x) • Multivariate (y,x

Predictive Causality

• In this case, we say that “x causes y”– It does not mean causality in a mechanical sense

– Only that  x  “predictively causes”  y

– True causality could actually be the reverse

• In economics, “predictive causality” is frequently called “Granger causality”

Page 26: Regression Models - University of Wisconsin–Madisonssc.wisc.edu/~bhansen/390/390Lecture18.pdf · 2014-04-01 · Regression Models • Bivariate data (y,x) • Multivariate (y,x

Clive Granger

• UCSD econometrician – 1934‐2009– Winner of 2003 Nobel Prize– Greatest time‐series econometrician of all time

– Photo from March 2007

• Many accomplishments– Granger causality– Spurious regression– Cointegration

Page 27: Regression Models - University of Wisconsin–Madisonssc.wisc.edu/~bhansen/390/390Lecture18.pdf · 2014-04-01 · Regression Models • Bivariate data (y,x) • Multivariate (y,x

Non‐Causality

• Hypothesis: – x does not predictively (Granger) cause y 

• Test– Reject hypothesis of non‐causality if joint test of all lags on x are zero

– F test using robust r option

tqtqt

ptptt

exx

yyy

++++

+++=

−−

−−

ββ

ααμ

L

L

11

11

Page 28: Regression Models - University of Wisconsin–Madisonssc.wisc.edu/~bhansen/390/390Lecture18.pdf · 2014-04-01 · Regression Models • Bivariate data (y,x) • Multivariate (y,x

STATA Command

• .reg t3   L(1/12).t3   L(1/12).t12,  r

• .testparm L(1/12).t12

Page 29: Regression Models - University of Wisconsin–Madisonssc.wisc.edu/~bhansen/390/390Lecture18.pdf · 2014-04-01 · Regression Models • Bivariate data (y,x) • Multivariate (y,x

T3 on T12

Page 30: Regression Models - University of Wisconsin–Madisonssc.wisc.edu/~bhansen/390/390Lecture18.pdf · 2014-04-01 · Regression Models • Bivariate data (y,x) • Multivariate (y,x

Lags on T12

Page 31: Regression Models - University of Wisconsin–Madisonssc.wisc.edu/~bhansen/390/390Lecture18.pdf · 2014-04-01 · Regression Models • Bivariate data (y,x) • Multivariate (y,x

Causality Test

• P‐value is near zero• Reject hypothesis of non‐causality• Infer that 12‐month T‐Bill helps predict 3‐month T‐bill• Long rates help to predict short rates

Page 32: Regression Models - University of Wisconsin–Madisonssc.wisc.edu/~bhansen/390/390Lecture18.pdf · 2014-04-01 · Regression Models • Bivariate data (y,x) • Multivariate (y,x

Reverse: T12 on T3

• Do short rates help to forecast long rates?

• Regress T12 on lagged values, and lags of T3

Page 33: Regression Models - University of Wisconsin–Madisonssc.wisc.edu/~bhansen/390/390Lecture18.pdf · 2014-04-01 · Regression Models • Bivariate data (y,x) • Multivariate (y,x

T12 on T3

Page 34: Regression Models - University of Wisconsin–Madisonssc.wisc.edu/~bhansen/390/390Lecture18.pdf · 2014-04-01 · Regression Models • Bivariate data (y,x) • Multivariate (y,x

Lags on T3

Page 35: Regression Models - University of Wisconsin–Madisonssc.wisc.edu/~bhansen/390/390Lecture18.pdf · 2014-04-01 · Regression Models • Bivariate data (y,x) • Multivariate (y,x

Causality Test

• P‐value is nearly significant• Not clear if we reject hypothesis of non‐causality• Unclear if 3‐month T‐Bill helps predict 12‐month T‐bill

– If short rates help to predict long rates

Page 36: Regression Models - University of Wisconsin–Madisonssc.wisc.edu/~bhansen/390/390Lecture18.pdf · 2014-04-01 · Regression Models • Bivariate data (y,x) • Multivariate (y,x

Term Structure Theory

• This is not surprising, given the theory of the term structure of interest rates

• Helpful to review interest rate theory

Page 37: Regression Models - University of Wisconsin–Madisonssc.wisc.edu/~bhansen/390/390Lecture18.pdf · 2014-04-01 · Regression Models • Bivariate data (y,x) • Multivariate (y,x

Bonds

• A bond with face value $1000 is a promise to pay $1000 at a specific date in the future– If that date is 3 months from today, it is a 3‐month bonds

– If that date is 12 months from today, it is a 12‐month bond

• Rate: If a 3‐month $1000 bond sells for $980, the interest percentage for the 3‐month period is 100*20/980=2.04%, or 8.16% annual rate

Page 38: Regression Models - University of Wisconsin–Madisonssc.wisc.edu/~bhansen/390/390Lecture18.pdf · 2014-04-01 · Regression Models • Bivariate data (y,x) • Multivariate (y,x

Term Structure

• Suppose an investor has a 2‐period horizon– They can purchase a 2‐period bond– Or a sequence of one‐period bonds

• Competitive equilibrium sets the prices of the bonds so they have equal expected returns.– The average expected one‐period returns equal the two‐period return

– The two‐period return is an expectation of future short rates

( )2

|1 tttt

ShortEShortLong Ω+= +

Page 39: Regression Models - University of Wisconsin–Madisonssc.wisc.edu/~bhansen/390/390Lecture18.pdf · 2014-04-01 · Regression Models • Bivariate data (y,x) • Multivariate (y,x

Term Structure Regression

• This implies

• Thus a predictive regression for short‐term interest rates is a function of lagged long‐term interest rates

• Long‐term interest rates help forecast short term rates because long‐term rates are themselves market forecast of future short rates– High long‐term rates mean that investors expect short rates to rise in the future

( ) tttt ShortLongShortE −=Ω+ 2|1

Page 40: Regression Models - University of Wisconsin–Madisonssc.wisc.edu/~bhansen/390/390Lecture18.pdf · 2014-04-01 · Regression Models • Bivariate data (y,x) • Multivariate (y,x

Causality

• The theory of the term structure predicts that long‐term rates will help predict short‐term rates

• It does not predict the reverse

• This is consistent with our hypothesis tests– 12‐month T‐Bill predicted 3‐month T‐Bill

– Unclear if 3‐month predicts 12‐month.

Page 41: Regression Models - University of Wisconsin–Madisonssc.wisc.edu/~bhansen/390/390Lecture18.pdf · 2014-04-01 · Regression Models • Bivariate data (y,x) • Multivariate (y,x

Selection of Causal Variables

• Even if we don’t reject non‐causality of y by x, we still might want to include x in forecast regression– Testing is not a good selection method

– AIC is a better for selection

Page 42: Regression Models - University of Wisconsin–Madisonssc.wisc.edu/~bhansen/390/390Lecture18.pdf · 2014-04-01 · Regression Models • Bivariate data (y,x) • Multivariate (y,x

Example

• Prediction of 3‐month rate– AR(12) only: AIC=‐1249– AR(12) plus T12(12 lags): AIC=‐1313– Full model has smaller AIC, so is preferred for forecasting

• This is consistent with causality test

• Prediction of 12‐month rate– AR(12) only: AIC=‐1309– AR(12) plus T3(12 lags): AIC=‐1333– Full model has smaller AIC, so is preferred for forecasting

• Even though we cannot reject non‐causality, AIC recommends using the short rate to forecast the long rate