Lesson 10: Covariances of causalARMA Processes

Umberto Triacca

Dipartimento di Ingegneria e Scienze dell’Informazione e MatematicaUniversita dell’Aquila,

umberto.triacca@univaq.it

Umberto Triacca Lesson 10: Covariances of causal ARMA Processes

A causal ARMA(p, q) process

Consider a causal ARMA(p, q) process

xt −φ1xt−1− ...−φpxt−p = ut + θ1ut−1 + ...+ θqut−q ∀t ∈ Z,

ut ∼ WN(0, σ2u)

φ(z) = 1− φ1z − ...− φpzp

andθ(z) = 1 + θ1z ... + θqz

q

Umberto Triacca Lesson 10: Covariances of causal ARMA Processes

A causal ARMA(p, q) process

The causality assumption implies that there exists constantsψ0, ψ1, ... such that

∞∑j=0

|ψj | <∞

and

xt =∞∑j=0

ψjut−j ∀t.

The sequence {ψ0, ψ1, ...} is determined by the identity

(1− φ1z − ...− φpzp) (ψ0 + ψ1z ...) = 1 + θ1z ... + θqz

q

Umberto Triacca Lesson 10: Covariances of causal ARMA Processes

A causal ARMA(p, q) process

Equating coefficients of z j , j=0,1,. . . , we obtain

ψ0 = 1

ψ1 = θ1 + ψ0φ1

ψ2 = θ2 + ψ1φ1 + ψ0φ2

and so on.

Umberto Triacca Lesson 10: Covariances of causal ARMA Processes

A causal ARMA(p, q) process

We have

ψj = θj +

p∑k=1

φkψj−k , j = 0, 1...,

where θ0 = 1, θj = 0 for j > q and ψj = 0 for j < 0.

Umberto Triacca Lesson 10: Covariances of causal ARMA Processes

The Autocovariance Function of a causal

ARMA(p, q) process

Since

E (xt) =∞∑j=0

ψjE (ut−j) = 0 ∀t.

We have

γ(k) = E (xtxt−k)

= E(∑∞

j=0 ψjut−j∑∞

i=0 ψiut−k−i)

=∑∞

j=0

∑∞i=0 ψjψiE (ut−jut−k−i)

Umberto Triacca Lesson 10: Covariances of causal ARMA Processes

The Autocovariance Function of a causal

ARMA(p, q) process

Since ut ∼ WN(0, σ2), we have that E (ut−jut−k−i) = σ2 ifi = j − k and 0 otherwise. Therefore

γ(k) = σ2∞∑j=0

ψjψj−k k = 0, 1, ...

Umberto Triacca Lesson 10: Covariances of causal ARMA Processes

The Autocovariance Function of a causal

ARMA(p, q) process

It is possible to show that the sequence {γ(k)}, is absolutelysummable, that is

∞∑k=−∞

|γ(k)| <∞.

Further, we observe that the absolute summability of thesequence {γ(k)} implies that

limk→∞

γ(k) = 0.

Umberto Triacca Lesson 10: Covariances of causal ARMA Processes

The Autocovariance Function of a causal

ARMA(p, q) process

We can summarize our findings about the autocovariancefunction of a causal ARMA(p, q) process as follows:

The autocovariance function of a causal ARMA(p, q)process is absolutely summable;

The autocovariance function of a causal ARMA(p, q)process vanishes when the lag tends to infinity.

Umberto Triacca Lesson 10: Covariances of causal ARMA Processes

Ergodicity and ARMA process

Are the causal ARMA processes ergodic?

Umberto Triacca Lesson 10: Covariances of causal ARMA Processes

Ergodic Theorems

Corollary 1 (Sufficient condition for mean-ergodicity). Let xtbe a stationary process with mean µ and autocovariancefunction γx(k). If

limk→∞γx(k) = 0,

then xt is mean-ergodic.

Corollary 2. (Sufficient condition for mean-ergodicity) Let xtbe a stationary process with mean µ and and autocovariancefunction γx(k). If

∞∑k=0

|γx(k)| <∞,

then xt is mean-ergodic.

Umberto Triacca Lesson 10: Covariances of causal ARMA Processes

Ergodicity under Gaussianity

Let {xt ; t ∈ Z} be a stationary process with mean µ andautocovariance function γx(k). If the process is Gaussian, then1. the condition of absolute summability of covariancefunction

∞∑k=0

|γx(k)| <∞

is sufficient to ensure ergodicity for all moments.2. the condition

limk→∞γx(k) = 0

is necessary and sufficient.

Umberto Triacca Lesson 10: Covariances of causal ARMA Processes

Ergodicity and ARMA process

Thus we have that

a causal ARMA process is ergodic for the mean.

a Gaussian causal ARMA process is ergodic for allmoments.

Umberto Triacca Lesson 10: Covariances of causal ARMA Processes

The Autocovariance Function of a causal

ARMA(1, 1) process

Consider a causal ARMA(1, 1) process:

xt − φxt−1 = ut + θut−1, ut ∼ WN(0, σ2)

Umberto Triacca Lesson 10: Covariances of causal ARMA Processes

The Autocovariance Function of a causal

ARMA(1, 1) process

We find that

ψ0 = 1 and ψj = (φ + θ)φj−1 for j ≥ 1.

Umberto Triacca Lesson 10: Covariances of causal ARMA Processes

The Autocovariance Function of a causal

ARMA(1, 1) process

Thus we have

γ(0) = σ2∑∞

j=0 ψ2j

= σ2[

1 + (φ + θ)2∑∞j=0 φ

2j]

= σ2[

1 + (φ+θ)2

1−φ2

]

γ(1) = σ2∑∞

j=0 ψjψj−1

= σ2[φ + θ + (φ + θ)2 φ

∑∞j=0 φ

2j]

= σ2[φ + θ + (φ+θ)2φ

1−φ2

]and

γ(k) = φk−1γ(1), k ≥ 2.

Umberto Triacca Lesson 10: Covariances of causal ARMA Processes

The Autocovariance Function of a causal

ARMA(p, q) process

The autocovariance function, γ(k), of a causal ARMA processtends to zero, as k →∞, at an exponential rate, that is, thereexist constants, D and δ such that, as k →∞, γ(k) ≈ Dδk ,with −1 < δ < 1

Umberto Triacca Lesson 10: Covariances of causal ARMA Processes

The Autocovariance Function of a causal AR(1)

process

Consider a causal AR(1) process:

xt − φxt−1 = ut , ut ∼ WN(0, σ2)

Umberto Triacca Lesson 10: Covariances of causal ARMA Processes

The Autocovariance Function of a causal AR(1)

process

We haveγ(0) = σ2

(1

1−φ2

)and

γ(k) = φkγ(0), k ≥ 1.

Umberto Triacca Lesson 10: Covariances of causal ARMA Processes

The Autocovariance Function of a causal AR(1)

process

Consider a causal AR(1) process:

xt − 0.7xt−1 = ut , ut ∼ WN(0, 1)

Umberto Triacca Lesson 10: Covariances of causal ARMA Processes

The Autocovariance Function of a causal AR(1)

process

Consider a causal AR(1) process:

xt + 0.7xt−1 = ut , ut ∼ WN(0, 1)

Umberto Triacca Lesson 10: Covariances of causal ARMA Processes

The Autocovariance Function of a causal AR(1)

process

Thus the autocovariances decay exponentially in one of twoforms.

1 If 0 < φ < 1, the autocovariances are positive;

2 If −1 < φ < 0 they oscillate between positive andnegative values.

Umberto Triacca Lesson 10: Covariances of causal ARMA Processes

The Autocovariance Function of a causal AR(2)

process

Consider a causal AR(2) process:

xt = φ1xt−1 + φ2xt−2 + ut , ut ∼ WN(0, σ2u)

The autocovariance function of this process is given by thefollowing recursive formula:

γx(k) = φ1γx(k − 1) + φ2γx(k − 2) k = 2, 3, ...

with the initial conditions

γx(0) =(1− φ2)σ2

u

(1 + φ2)[

(1− φ2)2 − φ21

]and

γx(1) =φ1γx(0)

(1− φ2).

Umberto Triacca Lesson 10: Covariances of causal ARMA Processes

The Autocovariance Function of a causal AR(2)

process

Consider a causal AR(2) process:

xt = 0.5xt−1 + 0.3xt−2 + ut , ut ∼ WN(0, 1)

Umberto Triacca Lesson 10: Covariances of causal ARMA Processes

The Autocovariance Function of a causal AR(2)

process

In general, the autocovariance function of AR(p) processesdecays exponentially.

Umberto Triacca Lesson 10: Covariances of causal ARMA Processes

The Autocovariance Function of an MA(1) process

Consider an MA(1) process:

xt = ut + θut−1, ut ∼ WN(0, σ2)

We haveγ(0) = σ2 (1 + θ2)

γ(1) = σ2θ

andγ(k) = 0, k ≥ 2.

Umberto Triacca Lesson 10: Covariances of causal ARMA Processes

The Autocovariance Function of a causal MA(q)

process

In general, the autocovariance function of MA(q) models iszero for all lags greater than q.

Umberto Triacca Lesson 10: Covariances of causal ARMA Processes

The Autocovariance Function of a causal

ARMA(p, q) process

Causal ARMA processes are short memory processes. Thismeans that their autocovariance function dies out quickly, andhence

∞∑k=−∞

|γ(k)| <∞

Umberto Triacca Lesson 10: Covariances of causal ARMA Processes

Long memory processes

A stationary process xt with an autocovariance function γ(k)is called a long memory process, if

∞∑k=−∞

|γ(k)| =∞

Umberto Triacca Lesson 10: Covariances of causal ARMA Processes

Conclusion

In this lecture, we have considered the calculation of theautocovariance function γ(·) of a causal ARMA process.We remenber that the autocorrelation function is readily foundfrom the autocovariance function on dividing by γ(0).The partial autocorrelation function is also found from thefunction γ(·).

Umberto Triacca Lesson 10: Covariances of causal ARMA Processes

Conclusion

We close this lecture by introducing the lag operator L.

Definition. Lxt = xt−1

Properties.

1 Lj = xt = xt−j2 L(φxt) = φLxt = φxt−1

3 LiLjxt = Lixt−j = xt−j−i = Li+jxt4 L(xt + yt) = Lxt + Lyt = xt−1 + yt−1

Umberto Triacca Lesson 10: Covariances of causal ARMA Processes

Conclusion

Finally note that by convention we define the zeroth power ofL to be the identity operator, i. e.

L0xt = xt

Thus we can consider polynomials of degree n ≥ 0 in the lagoperator L, defined as

a(L) = a0L0 + a1L + ... + anL

n

where a0, a1, ..., an are real or complex coefficients. We have

a(L)xt = a0xt + a1xt−1 + ... + anxt−n

Umberto Triacca Lesson 10: Covariances of causal ARMA Processes

Conclusion

It may shown that the algebra of the polynomials of degreen ≥ 0 in the lag operator L, over the field of the real orcomplex numbers (i.e. with real or complex coefficients) isisomorphic to the algebra of polynomials in the real or complexindeterminate, say z .

We can manipulate the polynomials a(L), b(L),... like thepolynomials a(z), b(z),...

Umberto Triacca Lesson 10: Covariances of causal ARMA Processes

Conclusion

ARMA(1,1) in terms of the lag operator L.

Consider an ARMA(1,1) process defined by the equation

xt − φxt−1 = ut + θut−1 ut ∼ WN(0, σ2u)

By using the lag operator this equation can be rewritten as

φ(L)xt = θ(L)ut

where φ(L) = 1− φL and θ(L) = 1 + θL

Umberto Triacca Lesson 10: Covariances of causal ARMA Processes

Conclusion

ARMA(p,q) in terms of the lag operator L.

We can writeφ(L)xt = θ(L)ut

whereφ(L) = 1− φ1L− φ2L

2 − ...− φ1Lp

andφ(L) = 1 + θ1L + θ2L

2 − ... + θqLq

.

Umberto Triacca Lesson 10: Covariances of causal ARMA Processes