STAD57 Time Series Analysis

Lecture 10

1

Best Linear Predictors

Want 1-step-ahead BLP ,for forecasting based on 1-step-ahead BLP,

is found by solving (for φnj’s) the system:

2

1 1

1

0, 1,...,

( ) ( ), 1, ,

k

nn n

n

njj

E X X X k n

k j k k n

1nX 1,..., nX X2

1 1 ][( ) ,minimizing MSE nn nX XE

1 11

nnn nj n jj

X X

Best Linear Predictors

3

1

1 2

1 2

1 2

1 2

( ) ( ), 1, ,

( 1) (1 1) (1 2) (1 ) (1)( 2) (2 1) (2 2) (2 ) (2)

( ) ( 1) ( 2) ( ) ( )

(0) (1) ( 1) (1

n

njj

n n nn

n n nn

n n nn

n n nn

k j k k n

k nk n

k n n n n n n

n

1 2

1 2

)(1) (0) ( 2) (2)

( 1) ( 2) (0) ( )

n n nn

n n nn

n

n n n

Best Linear Predictors

System can be written in matrix form as:

4

1

2

3

(0) (1) (2) ( 1) (1)(1) (0) (1) ( 2) (2)(2) (1) (0) ( 3) (3)

( 1) ( 2) ( 3) (0) ( )

n

n

n

nn

n

n nn

nnn

n n n n

φ γ=ΓΓ

n nφ γ

Best Linear Predictors

BLP coefficients φnj are given by:

Matrix Γn is symmetric & positive definite(provided γ(0)>0) → it is invertible, and we can solve for φn

1-step-ahead BLP can be written as:

where5

1nn n n n n Γ φ γ φ Γ γ

1 11

nnn nj n j nj

X X φ X

1 1[ ]n nX X X X

Best Linear Predictors

MSE of 1-step-ahead BLP is given by:

Proof:

6

2 11 1 1 ] (0)[( )n n

n n n nn nX XP E γ Γ γ

1n

nP

7

Example

Consider AR(2) model: Find φn for n=1,2,3,…

8

1 1 2 2t t t tX X X W

9

Best Linear Predictors

For any AR(p) model

and for n≥p, we have:

To prove this, just write

This is the BLP because:

10

1 1t t p t p tX X X W , 1, ,

0, 1, ,nj j

nj

j pj p n

1 11

pnn j n jj

X X

1 1 1 11

1 0, 1,...,

k k

k

pnn n n j n jj

n

E X X X E X X X

E W X k n

Durbin-Levinson Algorithm

To find φn for general ARMA model, we have to solve linear system: For large n, this can be very time consuming

(need to invert n×n matrix) Fortunately, there is an iterative algorithm for

solving the system which is a lot faster Algorithm takes advantage of

special structure of Γn, which is symmetric with equal diagonal elements (a.k.a. Toeplitz matrix)

11

1nn n n n n Γ φ γ φ Γ γ

(0) (1) (2)(1) (0) (1)(2) (1) (0)n

Γ

Durbin-Levinson Algorithm

The BLP equations:

can be solved iteratively (in n), as follows:

Algorithm progresses as: φ0 → φ1→ φ2→ φ3→…12

1

11 (0)

nn

n n

n n

n nP

φ Γ γγ Γ γ

000 1

1 11, 1,1 1

1 11, 1,1 1

1 21

1, 1,

0 & (0) 1

( ) ( ) ( ) ( )

(0) ( ) 1 ( )

1

, 1,2, , 1 ( 2)

Start with , and for set:

if

n nn k n kk k

nn n nn k n kk k

n nn n nn

nk n k nn n n k

P n

n n k n n k

k k

P P

k n n

Example

Use the Durbin-Levinson algorithm to find & based on

13

3φ3

4P (0), (1), (2), (3)

14

PACF

Have defined PACF of a TS to be given by:

where:

are the BLP’s of Xt+h & Xt, based on {Xt+1,…,Xt+h−1}

We now look relationship of PACF to φ’s, and how to calculate it

15

11 1( , ) (1),ˆ ˆ[( ), ( )], 2

&t t

hh t h t h t t

Cor X X

Cor X X X X h

1 1 2 2 1 1

1 1 2 2 1 1

ˆ

ˆt h t h t h h t

t t t h t h

X X X X

X X X X

PACF

First, note that is 1-step-ahead BLP of based on

Since the optimal coefficients are the same for any t (by stationarity), assume t=0 for simplicity

16

ˆt hX t hX

1 2 1t h t h tX X X X

1,1 1 1,2 2 1, 1 1ˆ

t h h t h h t h h h tX X X X

11,1 1 1, 1 1

ˆ , 2hh h h h h hX X X X h

11 1 0

0 0

( , ) (1),ˆ ˆ[( ), ( )], 2

&i.e. , and

hh h h

Cor X X

Cor X X X X h

PACF

Also note that , the BLP of based onis given by

Uses the same coefficients as in reverse order Easy to prove, since BLP of solves system:

which is the same as for BLP of

17

0X̂ 0X

1 2 1h hX X X X

0 1, 1 1 1, 2 2 1,1 1ˆ , 2h h h h h h hX X X X h

ˆhX

1

0 0 1,1

ˆ 0 ( ) ( ), 1, , 1k

h

h jj

E X X X k j k k h

0X

ˆhX

PACF

Thus, we can writewhere:

We can now show that

where is given by BLP of Use fact that , where

a,b are constant vectors, X,Y are random vectors, and Cov(X,Y) is covariance matrix of X,Y 18

1 0 1ˆ ˆ&h h hX X φ X φ X

1 2 1 1 1,1 1,2 1, 1,h h h h h h hX X X X φ

1 1, 1 1, 2 1,1 1& , i.e. the reverse of h h h h h h h φ φ

ˆ ˆ[( ), ( )], 2hh t h t h t tCor X X X X h

hh1

hhX

, ( , )Cov Cov a X b Y a X Y b

19

20

21

PACF

So, given autocovariance γ(h) / ACF ρ(h), we can find PACF φhh iteratively, using the Durbin-Levinson algorithm:

To get the sample PACF, just use algorithm with sample autocovariance / sample ACF

22

11 (1) 2& for h 1 1

1, 1,1 11 1

1, 1,1 1

1, 1,

( ) ( ) ( ) ( )

(0) ( ) 1 ( )

, 1,2, , 1 ( 2)if

h hh k h kk k

hh h hh k h kk k

hk h k hh h h k

h h k h h k

k k

k h h

Example

Find PACF of AR(2):

23

1 1 2 2t t t tX X X W