Introduction to Times Series

Introduction to Time Series Analysis. Lecture 4.Peter Bartlett

Last lecture:

1. Sample autocorrelation function

2. ACF and prediction

3. Properties of the ACF

1

Introduction to Time Series Analysis. Lecture 4.Peter Bartlett

1. Review: ACF, sample ACF.

2. Properties of estimates of and .

3. Convergence in mean square.

2

Mean, Autocovariance, Stationarity

A time series {Xt} has mean function t = E[Xt]and autocovariance function

X(t+ h, t) = Cov(Xt+h, Xt)= E[(Xt+h t+h)(Xt t)].

It is stationary if both are independent of t.Then we write X(h) = X(h, 0).The autocorrelation function (ACF) is

X(h) =X(h)

X(0)= Corr(Xt+h, Xt).

3

Estimating the ACF: Sample ACF

For observations x1, . . . , xn of a time series,

the sample mean is x = 1n

nt=1

xt.

The sample autocovariance function is

(h) =1

n

n|h|t=1

(xt+|h| x)(xt x), for n < h < n.

The sample autocorrelation function is

(h) =(h)

(0).

4

Properties of the autocovariance function

For the autocovariance function of a stationary time series {Xt},1. (0) 0,2. |(h)| (0),3. (h) = (h),4. is positive semidefinite.

Furthermore, any function : Z R that satisfies (3) and (4) is theautocovariance of some stationary (Gaussian) time series.

5

Introduction to Time Series Analysis. Lecture 4.1. Review: ACF, sample ACF.



6

Properties of the sample autocovariance function

The sample autocovariance function:

(h) =1

n

n|h|t=1

(xt+|h| x)(xt x), for n < h < n.

For any sequence x1, . . . , xn, the sample autocovariance function satisfies

1. (h) = (h),2. is positive semidefinite, and hence

3. (0) 0 and |(h)| (0).

7

Properties of the sample autocovariance function: psd

n =

(0) (1) (n 1)(1) (0) (n 2)

.

.

.

.

.

.

.

.

.

.

.

.

(n 1) (n 2) (0)

=1

nMM , (see next slide)

so ana =1

n(aM)(M a)

=1

nM a2

0,

i.e., n is a covariance matrix. It is also important for forecasting.

8

Properties of the sample autocovariance function: psd

M =

0 0 0 X1 X2 Xn0 0 X1 X2 Xn 00 X1 X2 Xn 0 0.

.

.

.

.

.

X1 X2 Xn 0 0

.

and Xt = Xt .

9

Estimating

How good is Xn as an estimate of ?

For a stationary process {Xt}, the sample average,

Xn =1

n(X1 + +Xn) satisfies

E(Xn) = , (unbiased)

var(Xn) =1

n

nh=n

(1 |h|

n

)(h).

10


To see why: var(Xn) = E

(1

n

ni=1

Xi ) 1

n

nj=1

Xj

=1

n2

ni=1

nj=1

E(Xi )(Xj )

=1

n2

i,j

(i j)

=1

n

n1h=(n1)

(1 |h|

n

)(h).

11

Estimating

Since var(Xn) =1

n

nh=n

(1 |h|

n

)(h),

if limh

(h) = 0, var(Xn) 0.

12

Estimating

Also, since var(Xn) =1

n

nh=n

(1 |h|

n

)(h),

ifh

|(h)|

Estimating

n var(Xn) 2

h=

(h).

i.e., instead of var(Xn) 2

n, we have var(Xn)

2

n/,

with =

h (h). The effect of the correlation is a reduction of samplesize from n to n/ . (c.f. mixing time.)

14

Estimating : Asymptotic distribution

Why are we interested in asymptotic distributions?

If we know the asymptotic distribution of Xn, we can use it toconstruct hypothesis tests,e.g., is = 0?

Similarly for the asymptotic distribution of (h),e.g., is (1) = 0?

Notation: Xn AN(n, 2n) means asymptotically normal:Xn n

n

d Z, where Z N(0, 1).

15

Estimating for a linear process: Asymptotically normal

Theorem (A.5) For a linear process Xt = +

j jWtj ,

ifj 6= 0, then

Xn AN(x,

V

n

),

where V =

h=

(h)

= 2w

j=

j

2

.

(X AN(n, n) means 1n (Xn n) d Z.)

16

Estimating for a linear process

Recall: for a linear process Xt = +

j jWtj ,

X(h) = 2w

j=

jh+j ,

so limn

n var(Xn) = limn

n1h=(n1)

(1 |h|

n

)(h)

= limn

2w

j=

j

n1h=(n1)

(j+h |h|

nj+h

)

= 2w

j=

j

2

.

17

Estimating the ACF: Sample ACF for White Noise

Theorem For a white noise process Wt,if E(W4

t)

Sample ACF and testing for white noise

If {Xt} is white noise, we expect no more than 5% of the peaks of thesample ACF to satisfy

|(h)| > 1.96n.

This is useful because we often want to introduce transformations thatreduce a time series to white noise.

19

Sample ACF for white Gaussian (hence i.i.d.) noise

20 15 10 5 0 5 10 15 200.2

0

0.2

0.4

0.6

0.8

1

1.2

20


Theorem (A.7) For a linear process Xt = +

j jWtj ,

if E(W4t

)

Sample ACF for MA(1)

Recall: (0) = 1, (1) = 1+2 , and (h) = 0 for |h| > 1. Thus,

V1,1 =h=1

((h+ 1) + (h 1) 2(1)(h))2 = ((0) 2(1)2)2 + (1)2,

V2,2 =h=1

((h+ 2) + (h 2) 2(2)(h))2 =1

h=1

(h)2.

And if is the sample ACF from a realization of this MA(1) process, thenwith probability 0.95,

|(h) (h)| 1.96Vhhn.

22

Sample ACF for MA(1)

10 8 6 4 2 0 2 4 6 8 100.2

0

0.2

0.4

0.6

0.8

1

1.2ACFConfidence intervalSample ACF

23

Introduction to Time Series Analysis. Lecture 4.1. Review: ACF, sample ACF.



24

Convergence in Mean Square

Recall the definition of a linear process:

Xt =

j=

jWtj

What do we mean by these infinite sums of random variables?i.e., what is the limit of a sequence of random variables?

Many types of convergence:1. Convergence in distribution.2. Convergence in probability.3. Convergence in mean square.

25

Convergence in Mean Square

Definition: A sequence of random variables S1, S2, . . .converges in mean square if there is a random variable Yfor which

limn

E(Sn Y )2 = 0

26

Example: Linear Processes

Xt =

j=

jWtj

Then if

j= |j |

Example: Linear Processes (Details)

(1) P (|Xt| ) 1

E|Xt| (Markovs inequality)

1

j=

|j |E|Wtj |

j=

|j | (Jensens inequality)

0.

28


For (2):The Riesz-Fisher Theorem (Cauchy criterion):Sn converges in mean square iff

limm,n

E(Sm Sn)2 = 0.

29


(2) Sn =n

j=n

jWtj converges in mean square, since

E(Sm Sn)2 = E m|j|n

jWtj

2

=

m|j|n

2j2

2 m|j|n

|j |

2

0.

30

Example: AR(1)

Let Xt be the stationary solution to Xt Xt1 =Wt, whereWt WN(0, 2).If || < 1,

Xt =j=0

jWtj

is a solution. The same argument as before shows that this infinite sumconverges in mean square, since || < 1 impliesj0 |j |

Example: AR(1)

Furthermore, Xt is the unique stationary solution: we can check that anyother stationary solution Yt is the mean square limit:

limn

E

(Yt

n1i=0

iWti

)2= lim

nE(nYtn)2

= 0.

32

Example: AR(1)

Let Xt be the stationary solution to

Xt Xt1 =Wt,

where Wt WN(0, 2).If || < 1,

Xt =

j=0

jWtj .

= 1? = 1?|| > 1?

33

Introduction to Time Series Analysis. Lecture 4.1. Properties of estimates of and .


34

Introduction to Times Series

Documents

Transcript of Introduction to Times Series