Multivariate State Space Models - Aarhus Universitet · Mulitvariate Kalman ﬁlter The Kalman...

Multivariate State Space Models

Siem Jan Koopman

http://staff.feweb.vu.nl/koopman

Department of Econometrics

VU University Amsterdam

Tinbergen Institute

2010

Multivariate State Space Models – p. 1

Multivariate local level model

Seemingly Unrelated Time Series Equations model:

yt = µt + εt, εt ∼ NID(0,Σε),

µt+1 = µt + ηt, ηt ∼ NID(0,Ση).

• Observations are p× 1 vectors;• The disturbances εt, ηs are independent for all s, t;• The p different time series are related through correlations in the

disturbances.

For a full discussion, see Harvey and Koopman (1997).

A pdf version (scanned) at http://staff.feweb.vu.nl/koopmanunder section “Publications” and subsection “Published articles ascontributions to books”.


Multivariate LL Model

The multivariate LL model is given by


µt+1 = µt + ηt, ηt ∼ NID(0,Ση).

• First difference∆yt = ηt−1 +∆εt

is stationary;• Reduced form: ∆yt is VMA(1) or VAR(∞);


Multivariate LL Model

• Stochastic properties are multivariate analogous of univariatecase:

Γ0 = E(∆yt ∆y′t) = Ση + 2Σε

Γ1 = E(∆yt ∆y′t−1) = −Σε

Γτ = E(∆yt ∆y′t−τ ) = 0, τ ≥ 2,

• The unrestricted vector MA(1) process has p2 + p(p+ 1)/2parameters, the SUTSE has p× (p+ 1);

• Such multivariate reduced form representations can also beestablished for general models.


Homogeneous Multivariate LL Model

The homogeneous multivariate LL model is given by


µt+1 = µt + ηt, ηt ∼ NID(0, qΣε),

where q is a non-negative scalar. This implies that Ση = qΣε.

• The model is restricted, all series in yt have the same dynamicproperties (the same acf).

• Not so relevant in practical work apart from forecasting. It is themodel representation for exponentially weighted moving average(EWMA) forecasting of multiple time series.

• This can be generalised to more general components models.• Easy to estimate, only a set of univariate Kalman filters are

required.


Common Levels

The common local level model is given by


µt+1 = µt + ηt, ηt ∼ NID(0,Ση),

where rank(Ση) = r < p.

• the model can be described by r underlying level components, thecommon levels;

•

Ση = AΣcA′,

A is p× r, Σc is r × r of full rank;• interpretation of A: factor loading matrix.


Common Levels

The common local level model


µt+1 = µt + ηt, ηt ∼ NID(0, AΣcA′),

can be rewritten in terms of underlying levels:

yt = a+Aµct + εt,

µct+1 = µc

t + ηct , ηct ∼ NID(0,Σc),

so thatµt = a+Aµc

t , ηt = Aηct .


Common Levels

For the common local level model


µt+1 = µt + ηt, ηt ∼ NID(0, AΣcA′),

notice that• decomposition Ση = AΣcA

′ is not unique;

• identification restrictions: Σc is diagonal, Choleski decomposition,principal compoments (based on economic theory);

• more interesting interpretation can be obtained by factor rotations;• can be interpreted as dynamic factor analysis, see later.


Common components

Common dynamic factors:• are useful for interpretation → cointegration;• have consequence for inference and forecasting (dimension of

parameter space reduces as a result).• common local level model can be generally represented as a

VAR(∞) or VECM models, details can be provided upon request.


Multivariate components

• So far, we have concentrated on multivariate variants of the locallevel model;

• Similar considerations can be applied to other components suchas the slope of the trend, seasonal and cycle components andother time-varying features in the multiple time series.

• Harvey and Koopman (1997) review such extensions.• In particular, they define the similar cycle component, see

Exercises.


Common and idiosyncratic factors

Multiple trends can also be decomposed into a one common factor andmultiple idiosyncratic factors:



where Ση = δδ′ +Dη with vector δ and diagonal matrix Dη. Thisimplies that the level can be represented by

µt = δµct + µ∗

t , ηt = δηct + η∗t

with common level (scalar) µct and ”independent” level µ∗

it generated by

∆µct+1 = ηct ∼ NID(0, 1), ∆µ∗

t+1 = η∗t ∼ NID(0, Dη).


Mulitvariate Kalman filter

The Kalman filter is valid for the general multivariate state spacemodel.

Computationally it is not convenient when p becomes large, verylarge.

Each step of the Kalman filter requires the inversion of the p× pmatrix Ft. This is no problem when p = 1 (univariate) but whenp > 20, say, it will slow down the Kalman filter considerably.

However, we can treat each element in the p× 1 observation vector ytas a single realisation. In other words, we can "update" eachsingle element of yt within the Kalman filter.

The arguments are given in DK book §6.4.

The same applies to smoothing.


Univariate treatment of Kalman filter

• Consider standard model: yt = Ztαt + εt and αt+1 = Ttαt +Rtηtwhere Var(εt) = Ht is diagonal.

• Observation vector yt = (yt,1, . . . , yt,pt)′ is treated and we view

observation model as a set of pt separate equations.• We then have, yt,i = Zt,iαt,i + εt,i with αt,i = αt for i = 1, . . . , pt.

• The associated transition equations become αt,i+1 = αt,i fori = 1, . . . , pt and αt+1,1 = Ttαt,pt

+Rtηt for t = 1, . . . , n.

• This disentangled model can be treated by the Kalman filter andsmoother equations straightforwardly.

• Innovations are now relative to the past and the “previous”observations inside yt,pt

!

• Non-diagonal matrix Ht can be treated by data-transformation orby including εt in the state vector αt.

• More details in DK book §6.4.


VAR representation of multivariate LL model

Consider the common local level (CLL) model as given by



where rank(Ση) = r < p.Rewrite CLL model as yt = a+ Aµc

t + εt and Ση = AΣcA′.

The relevant Kalman filter (KF) equations are

vt = yt − ct − Ztat, at+1 = Ttat +Ktvt, t = 1, . . . , n.

In case of CLL model: ct = a, Zt = A and µ̃ct|t−1 ≡ at.

The model can therefore be written in innovation form:

yt = a+ Aµ̃ct|t−1 + vt, vt ∼ NID(0, F ),

where µ̃ct|t−1 and vt are obtained from KF applied to the CLL model.


VAR(∞) Representation

The Kalman filter (in steady state) is

µ̃ct+1|t = µ̃c

t|t−1 +Kvt

= (I −KA)µ̃ct|t−1 +K(yt − a)

= [I − (I −KA)L]−1Kyt − (KA)−1Ka,

where L is lag operator and K is Kalman gain matrix.This leads to the alternative innovation form

yt = [I −A(KA)−1K]a

+A[I − (I −KA)L]−1KLyt + vt,

which is the VAR(∞) representation:

Φ(L)yt = Φ(1)a+ vt,

Φ(L) = I − A[I − (I −KA)L]−1KL.


VECM Representation

The VECM is based on decomposition

Φk(L) = Φ(1)L+Φ∗k−1(L)∆.

where Φ∗j coefficients are functions of Φk(L) coefficients.

This can be applied to the VAR(∞) representation of Common LLmodel:

Φ∗(L)∆yt = m+BC ′yt−1 + εt,

with singular matrix

Φ(1) = BC ′ = I −A(KA)−1K,

where B and C are N × r matrices with r = N − p and p is rank of Ση.Note that Φ(1)A = 0 and KΦ(1) = 0.


VAR(∞) Representation

• The VAR(∞) representation of the Common LL model isconsistent with a VECM of a cointegrating system;

• Short term dynamics are modelled differently, that is, thepolynomial Φ∗(L);

• Computations of VAR(∞) and VECM coefficients: Koopman&Harvey, JEDC 2003.


Illustration: US Monthly Housing Starts and Sold

Reinsel (1996) adopts bivariate cointegrated system for US monthlyhousing-starts and housing-sold (SA) for the period 1965 – 1974.He considers VARMA model

(I − ΦL)∆12yt = (I +ΘL12)ut,

where Θ is estimated as −I : seasonality is treated as deterministic.As VAR(1):

(I − Φ∗L)yt = m+ γt + ut,

with deterministic season γt.We use PcGive to obtain the VECM representation

∆yt = m+ γt + (−0.524, 0.141)′(1.00,−1.873)yt−1 + ut,

where we used Johansen’s cointegration test to conclude that matrixI − Φ has a rank close to 1.


US Monthly Housing Starts and Sold: PcGive output

----PcGive 10.0b session----

eigenvalue loglik for rank

-355.032 0

0.425082 -322.097 1

0.0182852 -320.999 2

Ho:rank=p -Tlog(1-\mu) using T-nm 95%

p == 0 65.87 ** 64.76 ** 14.1

p <= 1 2.196 2.159 3.8

standardized \beta’ eigenvectors

Hstarts Hsold

1.0000 -1.8730

0.52946 1.0000


US Monthly Housing Starts and Sold: PcGive output

standardized \alpha coefficients

Hstarts -0.52419 -0.027212

Hsold 0.14073 -0.023996

long-run matrix Po=\alpha * \beta’, rank 2

Hstarts Hsold

Hstarts -0.53860 0.95459

Hsold 0.12803 -0.28758


US Monthly Housing Starts and Sold: UC Model

In STAMPwe estimate the bivariate UC model:

yt = µt + γt + εt,

with µt as RW and γt as seasonal component.Estimation shows that seasonal component is fixed and a commonlevel exists:

µt = a+Aµct , µc

t+1 = µct + ηct .

Next we estimate common level model:

yt = (0.000, 2.581)′ + (1.00, 0.535)′µct + γt + εt,

with var(ηct ) estimated as 4.518.


US Monthly Housing Starts and Sold: STAMP output

----Stamp 6.20 session----

Summary statistics

Hstarts Hsold

Normality 0.41928 3.4801

Q( 9, 7) 9.1474 5.5896

Rsˆ2 0.18158 0.14433

Eq 1 : Estimated covariance matrices. (Lvl rank = two)

(upper-triangular = correlations)

Irr disturbance 11.095 -0.70665

-3.8864 2.7263

Lvl disturbance 19.388 0.96901

10.766 6.3667

Sea disturbance 0.00000 0.00000

0.00000 0.00000


US Monthly Housing Starts and Sold: STAMP output

Eq 2 : Estimated covariance matrices. (Lvl rank = one)

(upper-triangular = correlations)

Irr disturbance 12.778 -0.75708

-5.1727 3.6533

Lvl disturbance 20.411 1.0000

10.923 5.8456

Eq 2 : Diagonal and Load matrices.

Diag matrix Lvl 4.5178

Load matrix Lvl 1.0000 0.5352

Constant Lvl 0.0000 2.5807

Comparing results of PcGive (VECM analysis) and STAMP(UCanalysis) we observe that

C = (1.00,−1.873)′, A = (1.00, 0.535)′ → C ′A ≈ 0.

VECM and UC results are consistent with each other.


Illustration: the Euro business cycle

1985 1990 1995 2000

13.90

13.95

14.00

14.05

14.10

14.15

14.20

14.25


Topics in Business Cycle Analysis

• dating of business cycles (Markov-switching models)• prinicipal components analysis

(Stock and Watson, Forni, Hallin, Lippi and Reichlin)• convergence and synchronisation

(economic theory, empirical studies)• asymmetry and nonlinearities (econometrics)• coincident and leading indicators (economics)

In this illustration aim is to detect the business cycle• Detrending methods (Hodrick-Prescott);• Bandpass filtering methods (Baxter-King, Christiano-Fitzgerald);• Model-based, univariate (Beveridge-Nelson, Clark,

Harvey-Jaeger);• Model-based, multivariate, common cycles (VAR model, UC

model).


Different Univariate Trend-Cycle Decompositions

1985 1990 1995 2000

14.0

14.2

HP trend

1985 1990 1995 2000

−0.01

0.00

0.01

0.02

HP cycle

1985 1990 1995 2000

13.9

14.0

14.1

14.2

14.3

STAMP trend

1985 1990 1995 2000

−0.01

0.00

0.01

STAMP cycle

1985 1990 1995 2000

14.0

14.2

AKR trend

1985 1990 1995 2000

−0.01

0.00

0.01

AKR cycle


Univariate UC Trend-Cycle Decomposition

yt = µt + ψt + εt

• Trend µt: ∆dµt = ηt;• Irregular εt: White Noise;• Cycle ψt: AR(2) with complex roots as in Clark (1987) or with

stochastic trigonometric functions as in Harvey (1985,1989);

Trigonometric specification:

(ψt+1

ψ+t+1

)= φ

[cosλ sinλ

− sinλ cosλ

](ψt

ψ+t

)+

(κt

κ+t

),

κt, κ+t ∼NID(0, σ2

κ).

Signal extraction is about (locally) weighting observations. Kalmanfilter gives the optimal weights for the given models.


Weights and Gain Functions of Components

1985 1990 1995 2000

13.9

14.0

14.1

14.2

14.3

1985 1990 1995 2000

−0.02

0.00

0.02

−20 −10 0 10 20

0.0

0.1

0.2

−20 −10 0 10 20

0.0

0.5

0.0 0.2 0.4 0.6 0.8 1.0

0.5

1.0

0.0 0.2 0.4 0.6 0.8 1.0

0.5

1.0


Band-pass Properties

"Band-pass" refers to frequency domain properties of polynomial lagfunctions of time series (filters).

In business cycle analysis, one is interested in filters for trend andcycles such that trend only captures the low-frequencies, cycle themid-frequencies and irregular the high frequencies.

0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0

0.5

1.0

TREND

0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0

0.5

1.0

CYCLE

0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0

0.5

1.0

IRREGULAR


Butterworth Filters for Trend

Butterworth trend filters can be considered; they have a model-basedrepresentation and can be put in state space framework; see Gomez(JBES, 2001).

The m-th order stochastic trend is µt = µ(m)t where

∆mµ(m)t+1 = ζt, ζt ∼ NID(0, σ2

ζ ),

orµ(j)t+1 = µ

(j)t + µ

(j−1)t , “j = m,m− 1, . . . , 1,

with µ(0)t = ηt as before.

For m = 2 we have IRW with βt = µ(1)t .

Higher value for m gives low-pass gain function with sharper cut-offdownwards at certain low frequency point.


Generalised Cycle

Same principles can be applied to the cycle. The generalised kth order

cycle is given by ψt = ψ(k)t , where

(ψ(j)t+1

ψ+(j)t+1

)= φ

[cosλ sinλ

− sinλ cosλ

](ψ(j)t

ψ+(j)t

)+

(ψ(j−1)t

ψ+(j−1)t

),

j = 1, . . . , k,

with (ψ(0)t

ψ+(0)t

)=

(κt

κ+t ‘m

).

Higher orders ensure smoother transitions. Further details:Harvey & Trimbur (REStat, 2003).


Weights and Gain Functions of Components

1985 1990 1995 2000

13.9

14.0

14.1

14.2

14.3

1985 1990 1995 2000

−0.02

0.00

0.02

−20 −10 0 10 20

0.0

0.1

0.2

−20 −10 0 10 20

0.0

0.5

0.0 0.2 0.4 0.6 0.8 1.0

0.5

1.0

0.0 0.2 0.4 0.6 0.8 1.0

0.5

1.0


Measuring a common cycle from a multiple time series

• Analysis is based on a multivariate model• Data-set includes series that are leading, lagging GDP• We prefer not to choose leads & lags a-priori

• Common cycle will be allowed to shift for individual time seriesusing techniques developed by Rünstler (EctJ, 2004).

1980 1985 1990 1995

−0.4

−0.2

0.0

0.2

estimated cyclesgdp (red) versuscons confidence (blue)

1980 1985 1990 1995

−0.4

−0.2

0.0

0.2

estimated cyclesgdp (red) versusshifted cons confidence (blue)


Shifted cycles

In standard case, cycle ψt is generated by

(ψt+1

ψ+t+1

)= φ

[cosλ sinλ

− sinλ cosλ

](ψt

ψ+t

)+

(κt

κ+t

)

• The cyclecos(ξλ)ψt + sin(ξλ)ψ+

t ,

is shifted ξ time periods to the right (when ξ > 0) or to the left(when ξ < 0);

• Here, − 12π < ξ0λ <

12π (shift is w.r.t. ψt);

• More details in Rünstler (EctJ, 2004) for idea of shifting cycles inmultivariate unobserved components time series model;

• Also some details in paper of Koopman and Azevedo (2003), seehttp://staff.feweb.vu.nl/koopman ).


The basic multivariate model

Panel of N economic time series, yit, with basic model:

yit = µit + δiψt + εit, εit ∼ NID(0, σ2i,ε)

The time series have mixed frequencies (quarterly and monthly)

The final model is with shifted and generalised cycles:

yit = µ(k)it + δi

{cos(ξiλ)ψ

(m)t + sin(ξiλ)ψ

+(m)t

}+ εit.

Thus we have

• generalised individual trend µ(k)it ,

• generalised common cycle ψ(m)t with possible shifts;

• irregular εit.


Business cycle

Stock and Watson (1999):

“ . . . fluctuations in aggregate output are at the core of thebusiness cycle so the cyclical component of real GDP is auseful proxy for the overall business cycle . . . ”

We therefore impose a unit common cycle loading and zero phase shifton real GDP of the Euro area.

Time series 1986 – 2002:* quarterly GDP* industrial production* unemployment (countercyclical, lagging)* industrial confidence* construction confidence* retail trade confidence* consumer confidence* retail sales* interest rate spread (leading)


Eurozone Economic Indicators

1990 1995 2000

13.90

13.95

14.00

14.05

14.10

14.15

14.20

14.25

14.30 GDP IPI Interest rate spread Construction confidence indicator Consumer confidence indicator

Retail sales unemployment Industrial confidence indicator Retail trade confidence indicator


Details of model, estimation

• we have set m = 2 and k = 6 for generalised components• leads to estimated trend/cycle estimates with band-pass

properties, Baxter and King (1999).• frequency cycle is fixed at λ = 0.06545 (96 months, 8 years), see

Stock and Watson (1999) for the U.S. and ECB (2001) for theEuro area

• shifts ξi are estimated

• number of parameters for each equation is four (σ2i,ζ , δi, ξi, σ2

i,ε)

and for the common cycle is two (φ and σ2κ)

• total number is 4N = 4× 9 = 36


Decomposition of real GDP

1990 1995 2000

13.9

14.0

14.1

14.2

GDP Euro Area Trend

1990 1995 2000

0.001

0.002

0.003

slope

1990 1995 2000

−0.01

0.00

0.01

Cycle

1990 1995 2000

−0.0050

−0.0025

0.0000

0.0025

0.0050

irregular


The business cycle coincident indicator

Noteworthy features:• GDP is quarterly, estimated components are monthly• Euro area potential growth has declined after major recession of

1993 (before, growth was around 3.7% in annualised terms, after itwas 2.4%, falls within the 2.0− 2.5 underlying the ECB monetarypolicy)

• GDP cycle in line with common wisdom regarding Euro areabusiness cycle, ECB (2002)

• business cycle tracks the turning points well• historical minimum value is observed in Aug 1993, falls in most

severe recession period of Euro area• maximum value is in Jan 2001


The business cycle coincident indicator

Selected estimation results

series load shift R2d

gdp −− −− 0.31

indutrial prod 1.18 6.85 0.67

Unemployment −0.42 −15.9 0.78

industriual c 2.46 7.84 0.47

construction c 0.77 1.86 0.51

retail sales c 0.26 −0.22 0.67

consumer c 1.12 3.76 0.33

retail sales 0.11 −4.70 0.86

int rate spr 0.57 16.8 0.22


Coincident indicator for Euro area business cycle

1990 1995 2000

−0.015

−0.010

−0.005

0.000

0.005

0.010


Revisions

Real-time reliability of business cycle and growth indicators:• in practice, indicators are subject to revisions over time due to

data revisions and to their re-computation• not possible to evaluate consequences of first potential source of

revisions• we assess the second one by comparing smoothed and filtered

versions of indicator


Revisions

1990 1995 2000

−0.02

−0.01

0.00

0.01

0.02 Smoothed cycle Filtered cycle

1990 1995 2000

−0.02

−0.01

0.00

0.01

0.02revisions


Revisions, continued

Some revision statistics for cycle

period sd ratio corr sign

1989 – 2002 0.84 0.55 0.72

1993 – 2002 0.66 0.75 0.84

• take into account it is hard to estimate output gap in real-time• only with the increase of time, one can be more accurate about

cyclical position• Orphanides and van Norden (REStat, 2002) say whatever method

is used, reliability is quite low


Reliability statistics for business cycle indicators

Some comparisons for the quarterly frequency:

Method correlation noise-signal sign concord

86-02 93-02 86-02 93-02 86-02 93-02

H-P 0.33 0.70 1.23 0.99 0.47 0.55C-F 0.54 0.76 0.90 0.67 0.55 0.65H-C 0.42 0.69 0.91 0.75 0.61 0.63A-K-R 0.58 0.80 0.81 0.61 0.69 0.83

Correlation is the contemporaneous correlation between the real time(filtered) estimates and the final (smoothed) estimates of the businesscycle. Noise-to-signal ratio is the ratio of the standard deviation of therevisions against the standard deviation of the final estimates. Signconcordance is the percentage of times that the sign of the real timeand final estimates are equal.


Comparison of four different business cycle indicators

1985 1990 1995 2000

−0.015

−0.010

−0.005

0.000

0.005

0.010

0.015

0.020

H−P

C−F

H−CA−K−R


Filtered (dotted) and smoothed (solid) cycle estimates

1985 1990 1995 2000

−0.02

−0.01

0.00

0.01

0.02

H − P

1985 1990 1995 2000

−0.01

0.00

0.01

C − F

1985 1990 1995 2000

−0.01

0.00

0.01

H − C

1985 1990 1995 2000

−0.01

0.00

0.01

A − K − R


Dynamic factor models

Consider a basic example of the dynamic factor model, for p× 1observation vector yt and r × 1 latent factor vector ft, as given by

yt = Λft + εt, ft+1 = Φft + ζt, t = 1, . . . , n,

where εt ∼ NID(0,Σε) and ζt ∼ NID(0,Σζ). For identificationpurposes, we have vec(Σζ) = (I − Φ⊗ Φ)−1vec(I). In other words,factors ft are standardized.


The basic example

For p× 1 data vector yt and r factors in ft, the basic DFM is

yt = Λft + εt, ft+1 = Φft + ζt, t = 1, . . . , n.

Cross-section dimension p is typically high and time series length n ismoderate.

• We are possibly interested in p >> n.

• Estimation concentrates on Λ, Σε and Φ.

• However, first we concentrate on◦ signal extraction of ft,◦ likelihood evaluation,

for given values of Λ, Σε and Φ.


State space formulation

The dynamic factor model for p× 1 observation vector yt and with r × 1latent factor vector ft

yt = Λft + εt, ft+1 = Φft + ζt, t = 1, . . . , n,

and disturbances

εt ∼ NID(0,Σε) ζt ∼ NID(0,Σζ),

can be obviously represented in the familiar (partially time-invariant)state space model

yt = Zαt + εt, αt+1 = Ttαt +Rtηt,

withεt ∼ NID(0, H) ηt ∼ NID(0, Qt).

Let’s adopt the state space representation.


Signal extraction

Consider dynamic factor model


with high-dimensional yt and low-dimensional αt.

Likelihood evaluation can be based on predicion error decomposition

ℓ = p(y1)n∏

t=2

p(yt|y1, . . . , yt−1),

and can be routinely computed by the Kalman filter. Evaluation of

α̃t = E(αt|y1, . . . ys), V ar(αt|y1, . . . ys), s = t− 1, . . . , n,

for t = 1, . . . , n is then carried out by Kalman filter and related methods.

Kalman filter methods often dismissed as p becomes very large : (


Transformation by regression

However, huge computational gains can be obtained as follows : )Model


Apply GLS regression lemma for every t:

α̂t = Pyt, where P = (Z ′H−1Z)−1Z ′H−1.

Then, transform model for yt to a model for α̂t, that is

α̂t = αt + et,

with et = Pεt ∼ NID{0, (Z ′H−1Z)−1}. It can be shown that

α̃t = E(αt|y1, . . . , ys) = E(αt|α̂1, . . . , α̂s), t, s = 1, . . . , T.

It implies that observation equation dimension N reduces to r.


Two-step method

Modelyt = Zαt + εt, αt+1 = Ttαt +Rtηt,

for known system matrices.

Signal extraction for αt is carried out in two steps:

1. Cross-section step (GLS)

α̂t = (Z ′H−1Z)−1Z ′H−1yt.

2. Time series step: use Kalman filter methods to evaluateα̃t = E(αt|y1, . . . ys) based on low-dimensional model

α̂t = αt + et, et ∼ NID{0, (Z ′H−1Z)−1}

It turns out that all inference can be based on this model for α̂t,including the evaluation of the likelihood function.


Transforming the observation vector

Consider model yt = Zαt + εt with εt ∼ NID(0, H).

Transform y+t = Ayt, for t = 1, . . . , n, for some non-singular matrix A:MMSLEs are not affected and loglikelihood function differs only by theJacobian term log |A|n.

A =

[AL

AH

], y+t =

(yLtyHt

),

where yLt = ALyt, yHt = AHyt. Choose A s.t.

yLt = ALZαt + eLt , yHt = eHt ,

(eLteHt

)∼

{(0

0

),

[ΣL 0

0 ΣH

]}, with

ΣL = ALHAL′,

ΣH = AHHAH′.


Conditions for transformation

A suitable matrix A needs to fulfill the following conditions:

1. A is full rank, prevents any loss of information;

2. AHHAL′ = 0, ensures that both equations are independent;

3. Row{AH} = Col{Z}⊥ implies that yHt does not depend on αt (canbe weakened);

LEMMA 1:

Matrix A satisfies these conditions if and only if

AL = CZ ′H−1,

for any nonsingular r × r matrix C.

For this matrix AL, we can always find a matrix AH that satisfies 1,2,3.However, we will not need to compute AH , see below.


An additional condition for convenience

A suitable matrix A needs to fulfill the following conditions:

1. A is full rank, prevents any loss of information;

2. AHHAL′ = 0, ensures that both equations are independent;

3. Row{AH} = Col{Z}⊥ implies that yHt does not depend on αt;

4. |ΣH | = 1 where ΣH = AHHAH ′

The additional fourth condition is not restrictive, it is about scaling andit simplifies various calculations.

For example, from the fourth condition, it follows that

|A|2 = |H|−1|AHA′| = |H|−1|ALHAL ′||AHHAH ′| = |H|−1|ΣL|.

Particularly convenient for likelihood evaluation, next.


Likelihood evaluation

Gaussian likelihood (GL) based on transformation via A is

ℓ(y;ψ) = ℓ(yL;ψ) + ℓ(yH ;ψ) + n log |A|, |A|2 = |H|−1|ΣL|.

The first term ℓ(yL;ψ) can be evaluated by the Kalman filter.The second term is

ℓ(yH ;ψ) = −(p−m)n

2log 2π −

1

2

n∑

t=1

yH ′t Σ−1

H yHt ,

as the log-determinental term vanishes since |ΣH | = 1 (condition 4).

LEMMA 2:yH ′t Σ−1

H yHt = e′tH−1et,

where et = yt − Z(Z ′H−1Z)−1Z ′H−1yt is the GLS residual fordata-vector yt, covariate matrix Z and variance matrix H.Choice of C is irrelevant.


Likelihood evaluation

Gaussian likelihood (GL) can now be expressed as

ℓ(y;ψ) = c+ ℓ(yL;ψ)−n

2log

|H|

|ΣL|−

1

2

n∑

t=1

e′tH−1et,

where c is some constant, not dependent on both y and ψ.

It follows that for the evaluation of the loglikelihood, computation ofmatrix AH and vectors yHt , for t = 1, . . . , T , is not required.

Matrix H is oftentimes treated as diagonal or has other strongstructure (blocks, bands, spatial). Term |ΣL| is delivered by KFS.

This GL expression is instrumental for a computationally feasibleapproach to a quasi-likelihood based analysis of the dynamic factormodel.


Exercise 1

Consider the common trends model of Harvey and Koopman (1997,§§9.4.1 and 9.4.2).

1. Put the common trends model with (possibly common) stochasticslopes and based on equations (21)-(23) in state space form.

2. Put the common trends model with (possibly common) stochasticslopes and based on equations (24)-(26) in state space form.Define all vectors and matrices precisely.

3. Discuss the generalisation of Ση 6= 0 and the consequences forthe state space formulation of the model as in 2.


Exercise 2

Consider the multivariate trend model of Harvey and Koopman (1997).

1. Consider a multiple data set of N time series yt. The aim is todecompose the time series into trend and stationary components.It is further required that the multiple trend can be decomposedinto a common single trend (common to all N time series) andidiosyncratic trends (specific to the individual time series).

• Formulate a model for such a decomposition.• Discuss the identification of the different trends.• Express the model in state space form.

2. Once multiple trend models are expressed in state space form,we need to estimate the parameter coefficients of the model.Please describe shortly some relevant issues of maximumlikelihood estimation. Is it feasible ? What problems can youexpect ? Any recommendations for a successful implementation ?


Exercise 3

This exercise is based on the paper of Harvey and Koopman (1997),see my website.

Consider the similar cycle model of Harvey and Koopman (1997) withobservation equation

yt = ψt + εt, εt ∼ N(0,Σε),

where yt is a 3× 1 observation vector. Cycle ψt represents a commonsimilar cycle component of rank 2.

1. Please provide the state space representation of this model.

2. Comment on the restrictive nature of the similar cycle model.

3. How would you modify the similar cycle model so that each timeseries in yt has a different cycle frequency λ.

4. Can you apply the univariate Kalman filter of DK §6.4 in case Σε

is diagonal ? What if Σε is not diagonal ? Give details.


Exercise 4

Consider the dynamic factor model and the transformation approach.

Can you propose a transformation matrix A that applies to all threeconditions and that lead to a model for yLt with a diagonal variancematrix ΣL?

In this case, the univariate treatment of the Kalman filter can beconsidered.


Multivariate State Space Models - Aarhus Universitet · Mulitvariate Kalman ﬁlter The Kalman...

Documents

Transcript of Multivariate State Space Models - Aarhus Universitet · Mulitvariate Kalman ﬁlter The Kalman...