System Identification &

Parameter Estimation

Wb 2301, Lecture 12Validation & Parameter Accuracy

Erwin de Vlugt

Quantification of validity• Mathematical models:

– Variance-Accounted-For (VAF) values: How much of the variance in the data can be explained by the model?

∑

∑

=

=

−−= N

ii

N

iii

ty

tytyVAF

1

2

1

2

)(

))(ˆ)((1

data recorded:)u(t ),y(t);,()(ˆ

ii

tufty i θ=

Coherence and VAF

• High coherence, low VAF:– Linear system, good SNR, wrong model!

• High coherence, high VAF:– Linear system, good SNR, good model

• Low coherence, high VAF:– Non-linear system, good SNR, good non-linear

model• Low coherence, low VAF:

– Non-linear sytem or poor SNR, poor model

Accuracy of parameter fit

• Single parameter:– SEM: ‘Standard Error of the Mean’

• Multiple parameters:– Covariance matrix– Estimated from Jacobian and residual error

‘Standard error of the mean’(SEM)

• How accurate can the parameters be estimated?

• Example:– Normal distribution of data xN: μx, σx

– Standard Error of the Mean:

N

Nx

x

x

x

σσ

σσ

μ

μ

=

= 22 1variance of the mean

standard error (deviation) of the mean

Co-variance matrix

• Cov θN: Co-variance matrix of parameter vector θN

– σθN = √(diag(Cov θN))– Compare SEM

• Pθ: Co-variance matrix of distribution of error, given parameter vector θN

– Approximated by PN

– Compare σ2x (or better: σ2

e)

1]1.[1ˆ

1ˆ Cov

−=≈

≈

JJN

eeN

PP

PN

TTN

N

θ

θθ

Derivation PN

• ZN: data vector with input vector u and output vector y• VN(θ,ZN): criterion value

• θo: True, optimal parameter vector (unknown!)• Expanding Taylor series around θo:

),(minargˆ NNN ZV θθ

θ=

0),(),(' =∂

∂=

θθθ

NNNN

NNZVZV

No

N

NoN

NoN

oN

oN

NoN

NoN

PNN

ZVZV

ZVZV

N.1)ˆ(1

),(.)],([)ˆ(

)ˆ).(,(),(0

22ˆ

'1''

'''

=−=

−=−

−+=−

θθσ

θθθθ

θθθθ

θ

Derivation PN

• Where

• e is ‘white noise’, and hence ∂JT/∂θ = ∂2e/∂θ2 ≈ 0

TN

TNNNN VVVV

NP

−−= ''''1'' ...1

eJJJVV

eJVT

TNN

TN

.

.'

''

'

θθ ∂∂

+=∂∂

=

=

Derivation PN• PN becomes

• Where

1

11

11

''''1''

)1.(

)1.(1.)1.(

)1.(..11.)1(

...

−

−−

−−

−−

=

=

⎟⎠

⎞⎜⎝

⎛=

=

∑

JJN

JJN

JJN

JJN

JJN

JeeN

JN

JJN

VVVVP

TN

TTTN

TN

iii

TT

TN

TNNNN

λ

λ

eeN

eeN

TN

iiiN ..1.1== ∑λ

Co-variance matrix

• And σθ1 = √cov θN(1,1), etc.

⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢

⎣

⎡

=

2

2

2

21

2212

1211

..

..

..

ˆ Cov

MMM

M

M

N

θθθθθ

θθθθθ

θθθθθ

σσσσσ

σσσσσσσσσσ

θ