Autocorrelation Function Deﬁned Partial and...

Autocorrelation Function Properties and Examples

ρx(�) =γx(�)γx(0)

=γx(�)σ2

x

The ACF has a number of useful properties

• Bounded: −1 ≤ ρx(�) ≤ 1

• White noise, x(n) ∼WN(μx, σ2x): ρx(�) = δ(�)

• These enable us to assign meaning to estimated values fromsignals

• For example,

– If ρ̂x(�) ≈ δ(�), we can conclude that the process consists ofnearly uncorrelated samples

J. McNames Portland State University ECE 538/638 Autocorrelation Ver. 1.09 3

Partial and Autocorrelation Functions Overview

• Definitions

• Properties

• Yule-Walker Equations

• Levinson-Durbin recursion

• Biased and unbiased estimators


Example 1: 1st Order Moving Average

Find the autocorrelation function of a 1st order moving averageprocess, MA(1):

x(n) = w(n) + b1w(n− 1)

where w(n) ∼WN(0, σ2w).


Autocorrelation Function Defined

Normalized Autocorrelation, also known as the AutocorrelationFunction (ACF) is defined for a WSS signal as

ρx(�) =γx(�)γx(0)

=γx(�)σ2

x

where γx(�) is the autocovariance of x(n),

γxx(�) = E [[x(n + �)− μx][x(n)− μx]∗] = rx(�)− |μx|2


All-Pole Models

H(z) =b0

A(z)=

b0

1 +∑P

k=1 akz−k

• All-pole models are especially important because they can beestimated by solving a set of linear equations

• Partial autocorrelation can also be best understood within thecontext of all-pole models (my motivation)

• Recall that an AZ(Q) model can be expressed as an AP(∞)model if the AZ(Q) model is minimum phase

• Since the coefficients at large lags tend to be small, this can oftenbe well approximated by an AP(P ) model


Example 2: 1st Order Autoregressive

Find the autocorrelation function of a 1st order autoregressive process,AR(1):

x(n) = −a1x(n− 1) + w(n)

where w(n) ∼WN(0, σ2w). Hint: −αnu(−n− 1) Z←→ 1

1−αz−1 for an

ROC of |z| < |α|.


AP Equations

Let us consider a causal AP(P ) model:

H(z) +P∑

k=1

akH(z)z−k = b0

h(n) +P∑

k=1

akh(n− k) = b0δ(n)

P∑k=0

akh(n− k)h∗(n− �) = b0h∗(n− �)δ(n)

∞∑n=−∞

P∑k=0

akh(n− k)h∗(n− �) =∞∑

n=−∞b0h

∗(n− �)δ(n)

P∑k=0

akrh(�− k) = b0h∗(−�)


Autocorrelation Function Properties

ρx(�) =γx(�)γx(0)

=γx(�)σ2

x

• In general, the ACF of an AR(P ) process decays as a sum ofdamped exponentials (infinite extent)

• If the AR(P ) coefficients are known, the ACF can be determinedby solving a set of linear equations

• The ACF of a MA(Q) process is finite: ρx(�) = 0 for � > Q

• Thus, if the estimated ACF is very small for large lags a MA(Q)model may be appropriate

• The ACF of a ARMA(P, Q) process is also a sum of dampedexponentials (infinite extent)

• It is difficult to solve for in general


Solving the AP Equations

If we know the autocorrelation, we can solve these equations for a andb0 ⎡

⎢⎢⎢⎣

rh(0) r∗h(1) · · · r∗h(P )rh(1) rh(0) · · · r∗h(P − 1)

......

. . ....

rh(P ) rh(P − 1) · · · rh(0)

⎤⎥⎥⎥⎦

⎡⎢⎢⎢⎣

1a1

...aP

⎤⎥⎥⎥⎦ =

⎡⎢⎢⎢⎣

|b0|20...0

⎤⎥⎥⎥⎦


AP Equations Continued

Since AP(P ) is causal, h(0) = b0, h∗(0) = b∗0, and

P∑k=0

akrh(−k) = |b0|2 � = 0

P∑k=0

akrh(�− k) = 0 � > 0

This has several important consequences. One is that theautocorrelation can be expressed as a recursive relation for � > 0, sincea0 = 1:

P∑k=0

akrh(�− k) = 0

rh(�) = −P∑

k=1

akrh(�− k) � > 0


Solving for a

⎡⎢⎣

rh(1) rh(0) · · · r∗h(P − 1)...

.... . .

...rh(P ) rh(P − 1) · · · rh(0)

⎤⎥⎦

⎡⎢⎢⎢⎣

1a1

...aP

⎤⎥⎥⎥⎦ =

⎡⎢⎣

0...0

⎤⎥⎦

⎡⎢⎣

rh(1)...

rh(P )

⎤⎥⎦ +

⎡⎢⎣

rh(0) · · · r∗h(P − 1)...

. . ....

rh(P − 1) · · · rh(0)

⎤⎥⎦

⎡⎢⎣

a1

...aP

⎤⎥⎦ =

⎡⎢⎣

0...0

⎤⎥⎦

rh + Rha = 0a = −R−1

h rh

These are called the Yule-Walker equations


AP Equations in Matrix Form

We can collect the first P + 1 of these terms in a matrix⎡⎢⎢⎢⎣

rh(0) rh(−1) · · · rh(−P )rh(1) rh(0) · · · rh(−P + 1)

......

. . ....

rh(P ) rh(P − 1) · · · rh(0)

⎤⎥⎥⎥⎦

⎡⎢⎢⎢⎣

1a1

...aP

⎤⎥⎥⎥⎦ =

⎡⎢⎢⎢⎣

|b0|20...0

⎤⎥⎥⎥⎦

⎡⎢⎢⎢⎣

rh(0) r∗h(1) · · · r∗h(P )rh(1) rh(0) · · · r∗h(P − 1)

......

. . ....

rh(P ) rh(P − 1) · · · rh(0)

⎤⎥⎥⎥⎦

⎡⎢⎢⎢⎣

1a1

...aP

⎤⎥⎥⎥⎦ =

⎡⎢⎢⎢⎣

|b0|20...0

⎤⎥⎥⎥⎦

• The autocorrelation matrix is Hermitian, Toeplitz, and positivedefinite.


Yule-Walker Equation Comments Continued

a = R−1h rh b0 =

√rh(0) + aTrh

• Thus the two are equivalent and reversible and uniquecharacterizations of the model

{rh(0), . . . , rh(P )} ↔ {b0, a1, . . . , aP }• The rest of the sequence can then be determined by symmetry

and the recursive relation given earlier

rh(�) = −P∑

k=1

akrh(�− k) � > 0

rh(−�) = r∗h(�)


Solving for b0

⎡⎢⎢⎢⎣

rh(0) r∗h(1) · · · r∗h(P )rh(1) rh(0) · · · r∗h(P − 1)

......

. . ....

rh(P ) rh(P − 1) · · · rh(0)

⎤⎥⎥⎥⎦

⎡⎢⎢⎢⎣

1a1

...aP

⎤⎥⎥⎥⎦ =

⎡⎢⎢⎢⎣

|b0|20...0

⎤⎥⎥⎥⎦

[rh(0) r∗h(1) · · · r∗h(P )

]⎡⎢⎢⎢⎣

1a1

...aP

⎤⎥⎥⎥⎦ = |b0|2

b0 = ±√√√√ P∑

k=0

akrh(k)

= ±√

rh(0) + aTrh


AR Processes versus AP Models

Concisely, we can write the Yule-Walker Equations as

Rha = −rh

If we have an AR(P ) process, then we know rx(�) = σ2wrh(�) and we

can equivalently writeRxa = −rx

• Thus, the following two problems are equivalent

– Find the parameters of an AR process, {a1, . . . , aP , σ2w}, given

rx(�)– Find the parameters of an AP model, {a1, . . . , aP , b0}, given

rh(�)

• To accommodate both in a common notation, I will write theYule-Walker equations as simply

Ra = −r


Yule-Walker Equation Comments

a = R−1h rh b0 = ±

√rh(0) + aTrh

• The matrix inverse exists because unless h(n) = 0, Rh is positivedefinite

• Note that we cannot determine the sign of b0 = h(0) from rh(�)

• Thus, the first P terms of the autocorrelation completelydetermine the model parameters

• A similar relation exists for the first P + 1 elements of theautocorrelation sequence in terms the model parameters by solvinga set of linear equations (Problem 4.6)

• Is not true for AZ or PZ models


Partial Autocorrelation: Alternative Definition

Define P [x(n)|x(1), . . . , x(n− 1)] as the minimum mean square errorlinear predictor of x(n) given {x(1), . . . , x(n− 1)}

x̂(n) = P [x(n)|x(n− 1), . . . , x(1)] =n−1∑k=1

ckx(n− k)

where

ck = argminck

E[(x(n)− x̂(n))2

]

Similarly define P [x(0)|x(1), . . . , x(n− 1)] as the minimum meansquare error linear predictor of x(0) given {x(1), . . . , x(n− 1)},

x̂(0) = P [x(0)|x(n− 1), . . . , x(1)] =n−1∑k=1

dkx(n− k)


Solving for a Recursively

We can write the Yule-Walker equations as

⎡⎢⎢⎢⎣

r(0) r∗(1) · · · r∗(�− 1)r(1) r(0) · · · r∗(�− 2)

......

. . ....

r(�− 1) r(�− 2) · · · r(0)

⎤⎥⎥⎥⎦

⎡⎢⎢⎢⎢⎣

a(�)1

a(�)2...

a(�)�

⎤⎥⎥⎥⎥⎦ = −

⎡⎢⎢⎢⎣

r(1)r(2)

...r(�)

⎤⎥⎥⎥⎦

Ra = −r a = −R−1r

• We can recursively solve for the model coefficients

a = [a(�)1 , a

(�)2 , . . . , a

(�)� ] for increasing model orders

• Levinson-Durbin algorithm


Partial Autocorrelation: Alternative Definition & Properties

Then the PACF can be defined as the correlation between the residuals

x̃n(n) � x(n)− x̂1:n−1(n) = x(n)− P [x(n)|x(n− 1), . . . , x(1)]

x̃n(0) � x(0)− x̂1:n−1(0) = x(0)− P [x(0)|x(n− 1), . . . , x(1)]

α(�) � E [(x(�)− x̂n(�)) (x(0)− x̂n(0))]

E[(x(0)− x̂n(0))2

]

=E [(x(�)− x̂n(�)] (x(0)− x̂n(0))]

E[(x(n)− x̂n(n))2

]

• One can think of the PACF as a measure of the correlation ofwhat has not already been explained (the residuals)

• Like the ACF, it depends only on second order properties


Partial Autocorrelation

Partial Autocorrelation Function (PACF) also known as, the partialautocorrelation sequence (PACS), is defined as

α(�) �

⎧⎪⎨⎪⎩

1 � = 0a(�)� � > 0

α∗(−�) � < 0

where a(�)� is the last element of a ∈ R

� and is given by theYule-Walker equations

R︸︷︷︸�×�

a︸︷︷︸�×1

= − r︸︷︷︸�×1

• It is a dual of the ACF and has a number of useful andcomplimentary properties


Example 3: MA(1) ACF

0 1 2 3 4 5 6 7 8 9 10−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

ρ(l)

Lag (l)


Partial Autocorrelation Properties & Intuiting

α(�) �

⎧⎪⎨⎪⎩

1 � = 0a(�)� � > 0

α∗(−�) � < 0

• Intuitively you might expect |α(�)| < |ρ(�)|, but this is not true ingeneral

• Like ρ(�), the PACF is bounded: −1 ≤ α(�) ≤ 1

• White noise, x(n) ∼WN(0, σ2x): ρx(�) = δ(�)

• The PACF of a AR(P ) process is finite: αx(�) = 0 for � > P

• Thus, if the estimated PACF is very small for large lags a AR(P )model may be appropriate

• Surprisingly, the PACF is an infinite sequence for MA(Q)processes and ARMA(P, Q) processes


Example 3: MATLAB Code

L = 10; % Length of autocorrelation calculatedb1 = 0.9; % Coefficientsw = 1; % White noise powerac = sw*[(1+b1^2);b1;zeros(L-1,1)]; % Autocovariance = autocorrelation

l = 0:L;acf = ac/ac(1);h = stem(l,acf);set(h(1),’MarkerFaceColor’,’b’);set(h(1),’MarkerSize’,4);ylabel(’\rho(l)’);xlabel(’Lag (l)’);xlim([0 L]);ylim([-1 1]);box off;


Example 3: MA(1) ACF and PACF

Plot the ACF and PACF of a MA(1) model with b1 = 0.9. Hint: thetrue PACF is given by

α(�) =(−b1)�(1− b2

1)

1− b2(�+1)1


Example 3: Relevant MATLAB Code Continued

l = 0:L;h = stem(l,pc);set(h(1),’MarkerFaceColor’,’b’);set(h(1),’MarkerSize’,4);ylabel(’\alpha(l)’);xlabel(’Lag (l)’);xlim([0 L]);ylim([-1 1]);box off;


Example 3: MA(1) PACF

0 1 2 3 4 5 6 7 8 9 10−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

α(l)

Lag (l)


Example 4: AR(1) ACF and PACF

Plot the ACF and PACF of a AR(1) process with a = [1 0.9].


Example 3: Relevant MATLAB Code

pc = zeros(L+1,1);mc = zeros(L+1,1);pv = zeros(L+1,1);

pc(1) = 1;mc(1) = 1;pv(1) = ac(1);

pc(2) = ac(2)/ac(1);mc(2) = pc(2);pv(2) = ac(1)*(1-pc(2).^2);

for c1 = 3:L+1,pc(c1 ) = (ac(c1) - mc(2:c1-1).’*ac((c1-1):-1:2))/pv(c1-1);mc(2:c1-1) = mc(2:c1-1) - pc(c1)*mc(c1-1:-1:2);mc(c1 ) = pc(c1);pv(c1 ) = pv(c1-1)*(1-pc(c1).^2);end;


Example 4: AR(1) PACF

0 1 2 3 4 5 6 7 8 9 10−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

α(l)

Lag (l)


Example 4: AR(1) ACF

0 1 2 3 4 5 6 7 8 9 10−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

ρ(l)

Lag (l)


Autocovariance Estimation

• We’ve seen that the second-order statistics are a handy, thoughincomplete, characterization of WSS stochastic processes

• We would like to estimate these properties from realizations

– Single signal: γx(�), rx(�), αx(�), Rx(ejω)– Two or more signals: γyx(�), ryx(�), Ryx(�), G2

yx(ejω)

• What are the best estimators?


Example 4: Relevant MATLAB Code

L = 10; % Length of autocorrelation calculateda1 = 0.9; % Coefficientsw = 1; % White noise powerac = zeros(L+1,1);

ac(1) = sw/(1-a1^2);for c1=2:L+1,

ac(c1) = -a1*ac(c1-1);end;


Unbiased Autocovariance Estimation

γ̂u(�) � 1N − |�|

N−1−|�|∑n=0

[x(n + |�|)− μ̂x] [x(n)− μ̂x]

• If we used the true mean μx instead of μ̂x, γ̂u(�) would beunbiased

• When we use μ̂x the estimate is asymptotically unbiased

• The bias is O(1/N)

• Much smaller than the variance, so it may be ignored


Autocovariance Estimation Options

In practical applications, we only have a real finite data record{x(n)}N−1

0 . There are two popular estimators of autocovariance worthconsidering: “unbiased” and biased.

“Unbiased”

γ̂u(�) � 1N − |�|

N−1−|�|∑n=0

[x(n + |�|)− μ̂x] [x(n)− μ̂x] |�| < N

and γ̂u(�) = 0 for |�| ≥ N . Here μ̂x is the sample average of thesequence defined as

μ̂x � 1N

N−1∑n=0

x(n)


Biased Autocovariance Estimation

γ̂b(�) � 1N

N−1−|�|∑n=0

[x(n + |�|)− μ̂x] [x(n)− μ̂x] |�| < N

=N − |�|

Nγ̂u(�)

• Our book (and most other books) lists a different estimate

• This estimate uses a divisor of N rather than (N − |�|)• If we ignore the effect of estimating μx, this bias is obvious

E [γ̂(�)] =N − |�|

Nγ(�)

• The bias of this estimator is larger than the “unbiased” estimator

• Some claim that in general, the “biased” estimator has a smallerMSE

• The variance, must therefore be much smaller


Unbiased Autocovariance Estimation

γ̂u(�) � 1N − |�|

N−1−|�|∑n=0

[x(n + |�|)− μ̂x] [x(n)− μ̂x]

• Discussed briefly in the book

• The estimate has even symmetry: γ̂u(�) = γ̂u(−�)

• At longer lags, we have fewer terms to estimate the autocovariance

• We have no way to estimate γ(�) for |�| ≥ N

• We know that each pair {x(n + |�|), x(n)} for all n have the samedistribution because the process is assumed WSS and ergodic

• This is a natural estimator that we know converges asymptotically(N →∞)


Biased versus Unbiased Estimators

γ̂b(�) =N − |�|

Nγ̂u(�) ∝

N−1−|�|∑n=0

[x(n + |�|)− μ̂x] [x(n)− μ̂x]

• Although γ̂b(�) is biased,

– The bias is small at small lags

– For large lags, the bias is towards 0: γ̂(�)→ 0 as �→∞– This is also a property of the true autocorrelation

• If γ(�) is small for large lags, then the bias is also small

• The biased estimator has considerably less variance at large lags(the tail)

Biased var{γ̂b(�)} = O(1/N)

Unbiased var{γ̂u(�)} = O(1/(N − |�|))



γ̂b(�) =N − |�|

Nγ̂u(�) ∝

N−1−|�|∑n=0

[x(n + |�|)− μ̂x] [x(n)− μ̂x]

• The estimators are often called the sample autocovariancefunctions

• Most software and books prefer the biased estimate

• Why prefer a biased estimate to an unbiased estimate?

• Our goal is to estimate the sequence, not just γ(�) for a specificlag �


Biased is Better?

γ̂b(�) =N − |�|

Nγ̂u(�) ∝

N−1−|�|∑n=0

[x(n + |�|)− μ̂x] [x(n)− μ̂x]

• In general

– At small lags, there is little difference between the twoestimators

– At large lags, the larger bias of the biased model is favorablytraded for reduced variance

• In most cases, the biased model has smaller MSE, though it hasnot been proven rigorously

• For the remainder of the class will use the biased estimator, unlessotherwise noted γ̂(�) = γ̂b(�)



γ̂b(�) =N − |�|

Nγ̂u(�) ∝

N−1−|�|∑n=0

[x(n + |�|)− μ̂x] [x(n)− μ̂x]

• The key advantage of γ̂b(�) is that it is positive semi-definite (i.e.,nonnegative definite)

• There are many reasons why this property is important

– We know the true autocovariance has this property

– Autoregressive models built with the positive-definite estimatesof γ(�) are stable

– Most estimators of power spectral density R(ejω) arenonnegative if they are based on a positive-definite estimate ofγ(�)

• γ̂u(�) may be positive definite for a particular sequence, but it isnot guaranteed in general


Estimated Autocorrelation Variance Continued

If Gaussian random process,

var{r̂b(�)} =1N

N−�−1∑m=−(N+�)+1

(N−|m|+�

N

) (r2(m) + r(m + �)r(m− �)

)

• The same applies to the unbiased estimate with a divisor of1/(N − |�|) instead of 1/N

• This is still problematic because we don’t know the true r(�) inmost applications

• If we did, we wouldn’t need to estimate it!

• This is often what prevents us from making desired inferencesabout our estimators:

– Desired properties of the sampling distribution depend onunknown properties of the random process


Estimated Autocorrelation Covariance

γ̂b(�) =N − |�|

Nγ̂u(�) ∝

N−1−|�|∑n=0

[x(n + |�|)− μ̂x] [x(n)− μ̂x]

• As with all estimators, we would like to have confidence intervals

• These are hard to obtain, in general

• Need more assumptions

– Stationary up to order four

E [x(n)x(n + k)x(n + �)x(n + m)] = f(k, �, m)

– Mean is zero μx = 0, so does not need to be estimated


Estimated ACF

The natural estimate of the ACF is

ρ̂b(�) � γ̂b(�)γ̂(0)

ρ̂u(�) � γ̂u(�)γ̂(0)

• Same tradeoffs exist between the biased and unbiased estimates

• Also

– |ρ̂b(�)| ≤ 1 for all �

– Not true in general for ρ̂u(�)

• They are the same at � = 0

• Often called the sample autocorrelation function

• Again, the bias, covariance, and variance of the estimators iscomplicated and based on unknown properties


Estimated Autocorrelation Variance

γ̂b(�) = r̂b(�) =1N

N−1−|�|∑n=0

x(n + |�|)x(n)

γ̂u(�) = r̂u(�) =1

N − |�|N−1−|�|∑

n=0

x(n + |�|)x(n)

• The bias is

E [r̂b(�)] =N − |�|

Nr(�) E [r̂u(�)] = r(�)

• The covariance of r̂(�) is complicated and not usable in practice

– Depends on fourth joint cumulant of{x(n), x(n + k), x(n + �), x(n + m)}

– Depends on true unknown autocorrelation

• If process is Gaussian, then the fourth joint cumulant is zero


Confidence Intervals

• If N is large enough, the central limit theorem applies and ρ̂b(�) isapproximately normal

• In this case, we can use the Normal cdf to plot confidenceintervals of an IID sequence

• These are proportional to ±√var{ρ̂b(�)}


Estimated ACF Variance

Again, if x(n) is a Guassian process then

var{ρ̂b(�)} ≈ 1N

∞∑m=−∞

ρ2(m) + ρ(m + �)ρ(m− �)

+ 2ρ2(�)ρ2(m)− 4ρ(�)ρ(m)ρ(m− �)

• The fourth cumulant is also absent if x(n) is generated by a linearprocess with independent inputs

• The sample ACF, ρ̂(�) will generally have more correlation thanthe true ρ(�)

• It will generally be less damped and decay more slowly than ρ(�)

• Applies to the estimated autocovariance and autocorrelations aswell


Partial Autocorrelation Estimation

• There are similar issues surrounding partial autocorrelation

• However, in this case we always use the biased estimate ofautocorrelation to estimate the PACF

• This is necessary, in this case, to ensure that the AR models arebounded

• Less is known about the statistics of the PACF (mean, variance,and confidence intervals)

• However, for reasons similar to that of the ACF, for a WN processthe CLT applies and we can use the same confidence intervals asfor the ACF


Confidence Intervals

Let x(n) be an IID sequence. Then

ρ(0) = 1ρ(�) = 0 |�| > 0

cov{ρ̂(�), ρ̂(� + m)} ≈ 0 m �= 0

var{ρ̂b(�)} ≈ 1N

|�| > 0

var{ρ̂u(�)} ≈ N

(N − |�|)2 |�| > 0

• In general, it is not possible to obtain confidence intervals for theestimated ACF because the variance of the estimator depends onthe true ACF

• Instead, it is common practice to plot the confidence intervals of apurely random process



0 10 20 30 40 50 60 70 80 90−1

−0.5

0

0.5

1

Lag (l)

N=100 a1=−0.9

ρ(l)


Example 5: 1st Order Autoregressive

Find the autocorrelation function of a 1st order autoregressive process,AR(1):

x(n) = −a1x(n− 1) + w(n)

where w(n) ∼WN(0, σ2w). Estimate the ACF using the biased and

unbiased estimates for N = 100. Do so several times for differentvalues of a1.



0 10 20 30 40 50 60 70 80 90−1

−0.5

0

0.5

1

Lag (l)

N=100 a1=−0.9

ρ(l)


Example 5: AR(1) Signal

0 10 20 30 40 50 60 70 80 90 100

−1

0

1

2

3

4

5

6

Sample (n)

N=100 a1=−0.9

x(n)



0 10 20 30 40 50 60 70 80 90−1

−0.5

0

0.5

1

Lag (l)

N=100 a1=0.0

ρ(l)



0 10 20 30 40 50 60 70 80 90−1

−0.5

0

0.5

1

Lag (l)

N=100 a1=−0.9

α(l)



0 10 20 30 40 50 60 70 80 90−1

−0.5

0

0.5

1

Lag (l)

N=100 a1=0.0

ρ(l)



0 10 20 30 40 50 60 70 80 90 100

−2.5

−2

−1.5

−1

−0.5

0

0.5

1

1.5

2

Sample (n)

N=100 a1=0.0

x(n)



0 10 20 30 40 50 60 70 80 90−1

−0.5

0

0.5

1

Lag (l)

N=100 a1=0.5

ρ(l)



0 10 20 30 40 50 60 70 80 90−1

−0.5

0

0.5

1

Lag (l)

N=100 a1=0.0

α(l)



0 10 20 30 40 50 60 70 80 90−1

−0.5

0

0.5

1

Lag (l)

N=100 a1=0.5

ρ(l)



0 10 20 30 40 50 60 70 80 90 100

−2

−1

0

1

2

Sample (n)

N=100 a1=0.5

x(n)



0 10 20 30 40 50 60 70 80 90−1

−0.5

0

0.5

1

Lag (l)

N=100 a1=0.9

ρ(l)



0 10 20 30 40 50 60 70 80 90−1

−0.5

0

0.5

1

Lag (l)

N=100 a1=0.5

α(l)



0 10 20 30 40 50 60 70 80 90−1

−0.5

0

0.5

1

Lag (l)

N=100 a1=0.9

ρ(l)



0 10 20 30 40 50 60 70 80 90 100

−5

0

5

Sample (n)

N=100 a1=0.9

x(n)


Summary

• ACF and PACF are useful characterizations of WSS randomprocesses

• Can help select an appropriate model

– MA: Finite ACF

– AR: Finite PACF

• AP/AR are often preferred characterizations because we cansolve/estimate the model parameters by solving a set of linearequations (Yule-Walker)

• Biased estimates of r(�), ρ(�), and/or γ(�) are generally preferredto the unbiased estimates

– Less variance (always), and lower MSE (sometimes)

– Positive definite (PSD is therefore also nonnegative)

• Bias is known, but variance of estimates is generally unknown

• Loosely, confidence intervals for WN are used instead



0 10 20 30 40 50 60 70 80 90−1

−0.5

0

0.5

1

Lag (l)

N=100 a1=0.9

α(l)


Example 5: MATLAB Code

L = 90; % Length of autocorrelation calculateda1 = 0.9; % Coefficientsw = 1; % White noise powerac = zeros(L+1,1);N = 100;cl = 99; % Confidence levelnp = norminv((1-cl/100)/2); % Find corresponding lower percentileac(1) = sw/(1-a1^2);for c1=2:L+1,

ac(c1) = -a1*ac(c1-1);end;

acf = ac/ac(1);w = randn(N,1);a = [1 a1];x = filter(1,a,w);


Autocorrelation Function Deﬁned Partial and...

Documents

Transcript of Autocorrelation Function Deﬁned Partial and...