Introduction to Stochastic Processes -...

Introduction to Stochastic Processes

Eduardo RossiUniversity of Pavia

October 2013

Rossi Introduction to Stochastic Processes Financial Econometrics - 2013 1 / 28

Stochastic Process

Stochastic ProcessA stochastic process is an ordered sequence of random variables defined on aprobability space (Ω,F ,P).

Yt(ω), ω ∈ Ω, t ∈ T , such that for each t ∈ T , Yt(ω) is a random variable onthe sample space Ω, and for each ω ∈ Ω, Yt(ω) is a realization of the stochasticprocess on the index set T (that is an ordered set of values, each corresponds to avalue of the index set).

Time Series

A time series is a set of observations Yt , t ∈ T0, each one recorded at aspecified time t. The time series is a part of a realization of a stochastic process,Yt , t ∈ T where T ⊇ T0. An infinite series Yt∞t=−∞.


Stochastic Process

A statistical time series is a mathematical sequence, not a series.

Yt is assumed to be real valued, but we shall also consider sequences ofrandom vectors Yt, or variables with values in the complex numbers.

We are interested in the joint distribution of the variables Yt. The easiestway to ensure that these variables possess joint laws, is to assume that theyare defined as measurable maps on a single underlying probability space.

One of the purposes of time series analysis is to study and characterizeprobability distributions of sets of variables Yt that will typically bedependent.


Stochastic Process

The statistical problem is to determine the probability distribution of the timeseries given observations Y1,Y2, . . . ,YT .

The statistical model can be used for1 understanding the stochastic system;2 predicting the ’future’, i.e. YT+1,YT+2, . . .

In order to have any chance of success at these tasks it is necessary toassume some apriori structure of the time series.

If the Yt could be completely arbitrary random variables, thenY1, . . . ,YT would constitute a single observation from a completelyunknown distribution on RT .


Mean, Variance and Autocovariance

The unconditional mean

µt = E [Yt ] =

∫Yt f (Yt)dYt

Autocovariance function: The joint distribution of

(Yt ,Yt−1, . . . ,Yt−h)

is usually characterized by the autocovariance function:

γt(h) = Cov(Yt ,Yt−h)

= E [(Yt − µt)(Yt−h − µt−h)]

=

∫. . .

∫(Yt − µt)(Yt−h − µt−h)f (Yt , . . . ,Yt−h)dYt . . . dYt−h

The autocorrelation function

ρt(h) =γt(h)√

γt(0)γt−h(0)


Mean, Variance and Autocovariance

Both functions are symmetric about 0.

The autocovariance and autocorrelations functions are measures of (linear)dependence between the variables at different time instants.


Stationarity

A basic type of structure is stationarity. This comes in two forms.

Strict Stationarity

The process is said to be strictly stationary if for any values of h1, h2, . . . , hn thejoint distribution of (Yt ,Yt+h1 , . . . ,Yt+hn) depends only on the intervalsh1, h2, . . . , hn but not on the date t itself:

f (Yt ,Yt+h1 , . . . ,Yt+hn) = f (Yτ ,Yτ+h1 , . . . ,Yτ+hn) ∀t, h

Strict stationarity implies that all existing moments are time invariant.


Stationarity

Weak (Covariance) Stationarity

The process Yt is said to be weakly stationary or covariance stationary if thesecond moments of the process are time invariant:

E [Yt ] = µ <∞ ∀t

E [(Yt − µ)(Yt−h − µ)] = γ(h) <∞ ∀t, h

Stationarity implies γt(h) = γt(−h) = γ(h).

A strictly stationary time series with finite second moments is also stationary.

Gaussian Process

The process Yt is said to be Gaussian if the joint density of (Yt ,Yt+h1 , . . . ,Yt+hn),f (Yt ,Yt+h1 , . . . ,Yt+hn), is Gaussian for any h1, h2, . . . , hn.


Ergodicity

The statistical ergodicity theorem concerns what information can be derived froman average over time about the common average at each point of time.Note that the WLLN does not apply as the observed time series represents justone realization of the stochastic process.

Ergodic for the mean

Let Yt(ω), ω ∈ Ω, t ∈ T be a weakly stationary process, such that

E [Yt(ω)] = µ <∞ and E [(Yt(ω)− µ)2] = σ2 <∞ ∀t. Let yT = T−1∑T

t=1 Yt

be the time average. If yT converges in probability to µ as T →∞, Yt is said tobe ergodic for the mean.


Ergodicity

To be ergodic the memory of a stochastic process should fade in the sensethat the covariance between increasingly distant observations converges tozero sufficiently rapidly.

For stationary process it can be shown that absolutely summableautocovariances, i.e.

∑∞h=0 |γ(h)| <∞, are sufficient to ensure ergodicity.

Ergodic for the second moments

γ(h) = (T − h)−1T∑

t=h+1

(Yt − µ)(Yt−h − µ)P→ γ(h)

Ergodicity focus on asymptotic independence, while stationarity on thetime-invariance of the process.


Example

Deterministic trigonometric series:

Yt = A cos (tλ) + B sin (tλ)

E [A] = E [B] = 0, E [A2] = E [B2] = σ2 defines a stationary time series.

E [Yt ] = E [A cos (tλ) + B sin (tλ)] = 0

The Autocovariance function

γ(h) = E [YtYt+h]

= cos (t + h)λ cos (tλ) Var[A] + sin (t + h)λ sin (tλ) Var[B]

= σ2 cos (hλ)

Once A and B have been determined the process behaves as a deterministictrigonometric function. This type of time series is an important building block tomodel cyclic events in a system.


Example

Consider the stochastic process Yt defined by

Yt =

u0 t = 0 with u0 ∼ N(0, σ2)Yt−1 t > 0

Then Yt is strictly stationary but not ergodic.Proof Obviously we have that Yt = u0 for all t ≥ 0. Stationarity follows from:

E [Yt ] = E [u0] = 0

E [Y 2t ] = E [u2

0 ] = σ2

E [YtYt−1] = E [u20 ] = σ2 (1)


Example

Thus we have µ = 0, γ(h) = σ2 ρ(h) = 1 are time invariant. Ergodicity for themean requires:

yT = T−1T−1∑t=o

Yt = T−1(Tu0)

yT = u0


Random walk

Yt = Yt−1 + ut

where ut ∼WN(0, σ2). By recursive substitution,

Yt = Yt−1 + ut

Yt = Yt−2 + ut−1 + ut

...

= Y0 + ut + ut−1 + ut−2 + . . .+ u1

The mean is time-invariant:

µ = E [Yt ] = E

[Y0 +

t∑s=1

us

]= Y0 +

t∑s=1

E [us ] = 0.


Random walk

But the second moments are diverging. The variance is given by:

γt(0) = E [Y 2t ] = E

(Y0 +t∑

s=1

us

)2 = E

( t∑s=1

us

)2

= E

[t∑

s=1

t∑k=1

usuk

]= E

t∑s=1

u2s +

t∑s=1

t∑k=1,k 6=s

usuk

=

t∑s=1

E [u2s ] +

t∑s=1

t∑k=1,k 6=s

E [usuk ] =t∑

s=1

σ2 = tσ2.


Random walk

The autocovariances are:

γt(h) = E [YtYt−h] = E

[(Y0 +

t∑s=1

us

)(Y0 +

t−h∑k=1

uk

)]

= E

[t∑

s=1

us

(t−h∑k=1

uk

)]

=t−h∑k=1

E [u2k ]

=t−h∑k=1

σ2 = (t − h)σ2 ∀h > 0.

Finally, the autocorrelation function ρt(h) for h > 0 is given by:

ρ2t (h) =γ2t (h)

γt(0)γt−h(0)=

[(t − h)σ2]2

[tσ2][(t − h)σ2]= 1− h

t∀h > 0


White Noise Process

A white-noise process is a weakly stationary process which has zero mean and isuncorrelated over time:

ut ∼WN(0, σ2)

Thus ut is WN process ∀t ∈ T :

E [ut ] = 0

E [u2t ] = σ2 <∞

E [utut−h] = 0 h 6= 0, t − h ∈ T

If the assumption of a constant variance is relaxed to E [u2t ] <∞, sometimes

ut is called a weak WN process.

If the WN process is normally distributed it is called a Gaussian white-noiseprocess:

ut ∼ NID(0, σ2). (2)


The assumption of normality implies strict stationarity and serial independence(unpredictability). A generalization of the NID is the IID process with constant,but unspecified higher moments.A process ut with independent, identically distributed variates is denoted IID:

ut ∼ IID(0, σ2) (3)


Martingale

Martingale The stochastic process xt is said to be martingale with respect to aninformation set (σ-field), It−1, of data realized by time t − 1 if

E [|xt |] < ∞E [xt |It−1] = xt−1 a.s. (4)

Martingale Difference Sequence The process ut = xt − xt−1 with E [|ut |] <∞ andE [ut |It−1] = 0 for all t is called a martingale difference sequence, MDS.


Martingale difference

Let xt be an m.d. sequence and let gt−1 = σ(xt−1, xt−2, . . .) be anymeasurable, integrable function of the lagged values of the sequence. Then xtgt−1is a m.d., and

Cov [gt−1, xt ] = E [gt−1xt ]− E [gt−1]E [xt ] = 0

This implies that, putting gt−1 = xt−j ∀j > 0

Cov [xt , xt−j ] = 0

M.d. property implies uncorrelatedness of the sequence.White Noise processes may not be m.d. because the conditional expectation of anuncorrelated process can be a nonlinear.An example is the bilinear model (Granger and Newbold, 1986)

xt = βxt−2εt−1 + εt , εt ∼ i .i .d(0, 1)

The model is w.n. when 0 < β < 1√2

.


The conditional expectations are the following nonlinear functions:

E [xt |It−1] = −∞∑i=1

(−β)i+1xt−i

i+1∏j=2

xt−j

Uncorrelated, zero mean processes:

White Noise Processes1 IID processes

1 Gaussian White Noise processes

Martingale Difference Processes


Innovation

Innovation

An innovation ut against an information set It−1 is a process whose densityf (ut |It−1) does not depend on It−1.

Mean Innovation

ut is a mean innovation with respect to an information set It−1 ifE [ut |It−1] = 0.


The Wold Decomposition

If the zero-mean process Yt is wide sense stationary (implying E (Y 2t ) <∞) it has

the representation

Yt =∞∑j=0

ψjεt−j + vt

where ψ0 = 1,∑∞

j=0 ψ2j <∞

εt ∼WN(0, σ2)

εt = Yt − E [Yt |Yt−1,Yt−2, . . .]

E (vtεt−j) = 0,∀j

vt is perfectly predictable one-step ahead

vt = E [vt |Yt−1,Yt−2, . . .]

vt , linear deterministic component

If vt = 0, Yt is called a purely non-deterministic process.


Causal process

Linear Filter

The linear filter with filter coefficients (ψj : j ∈ Z) is the operation that transformsa given time series Xt into the time series

Yt =∑j∈Z

ψjXt−j

A linear filter is a moving average of infinite order. All filters used in practice arefinite filters: only finitely many coefficients ψj are nonzero. Important examplesare

the difference filter ∆Yt

Seasonal filter ∆kYt for k the ’length of the season’ or ’period’.


Causal process

Causal process

A linear process Yt is causal (strictly, a causal function of εt) if there is a

ψ(L) = ψ0 + ψ1L + ψ2L2 + . . .

with∑∞

j=0 |ψj | <∞

Yt = ψ0εt + ψ1εt−1 + ψ2εt−2 + . . .


Invertibility

Invertibility

A linear process Yt is invertible (strictly, an invertible function of εt) if thereis a

π(L) = π0 + π1L + π2L2 + . . .

with∞∑j=0

|ψj | <∞

andεt = π(L)Yt


Box-Jenkins approach to modelling time series

Approximate the infinite lag polynomial with the ratio of two finite-orderpolynomials φ(L) and θ(L):

ψ(L) =∞∑j=0

ψjLj ∼=

θ(L)

φ(L)=

1 + θ1L + θ2L2 + . . .+ θqLq

1− φ1L− φ2L2 . . .− φpLp

Time Series Models

p q Model Typep > 0 q = 0 φ(L)Yt = εt AR(P)p = 0 q > 0 Yt = θ(L)εt MA(Q)p > 0 q > 0 φ(L)Yt = θ(L)εt ARMA(p,q)


Invertible process

Yt has a causal ARMA process representation

Yt = ψ(L)εt =∑j

ψjYt−j

if ψj = 0, j < 0.

Yt has an invertible ARMA process representation

εt = π(L)Yt =∑j

πjYt−j

if the filter π(L) is causal, i.e. πj = 0, j < 0.


Introduction to Stochastic Processes -...

Documents

Transcript of Introduction to Stochastic Processes -...