# Bayesian Methods in Positioning Applications

of 21
/21

Embed Size (px)

### Transcript of Bayesian Methods in Positioning Applications

Vedran Dizdarevic

24. May 2006

Bayesian Methods in Positioning Applications – p.2/21

GRAZ UNIVERSITY OF TECHNOLOGY

Advanced Signal Processing Seminar

Problem Statement (1/2)

System description in two steps Find system model (modeling) Find the particular realization of the model

(identification; parameter estimation θ)

Time variant system parameters require recursive estimation on-line vs. off-line data processing

Bayesian Methods in Positioning Applications – p.3/21

GRAZ UNIVERSITY OF TECHNOLOGY

Advanced Signal Processing Seminar

With N-dimensional observation zk = {z1 k, . . . , z

N k }

Initial condition(s) - selection of θ0

Bayesian Methods in Positioning Applications – p.4/21

GRAZ UNIVERSITY OF TECHNOLOGY

Advanced Signal Processing Seminar

RLS

systems Extension of linear regression (LS) methods

Bayesian inference Estimation of distribution of θ

Bayesian Methods in Positioning Applications – p.5/21

GRAZ UNIVERSITY OF TECHNOLOGY

Advanced Signal Processing Seminar

Introduction to Bayesian Estimation

variable with p(θ)

of p(θ)

Relying on a Bayes rule updated p(θ) isinferredd from prior (predicted) p(θ) combined with incoming observations

Bayesian Methods in Positioning Applications – p.6/21

GRAZ UNIVERSITY OF TECHNOLOGY

Advanced Signal Processing Seminar

Bayes Rule and Contemporary Science

Published in T. Bayes Essay Towards Solving a Problem in the Doctrine of Chances (1764)

Steam engine of J. Watt (1769)

B. Franklin defines the concept of positive and negative polarity (1752)

L. Galvani and frogs (1770)

K.F. Gauss was born (1777)

Bayesian Methods in Positioning Applications – p.7/21

GRAZ UNIVERSITY OF TECHNOLOGY

Advanced Signal Processing Seminar

Bayes Rule

Assume a disjoint set of n discrete events {Ei} n i=1

Probabilities P (Ei) n i=1 are assigned to each event

If events are not independent observing that event Ek

hasoccurredd, probabilities of all other events {Ei} n i=1

will change

This is described using conditional probability P (Ei|Ek)

P (Ei, Ek) = P (Ei|Ek)P (Ek) = P (Ek|Ei)P (Ei)

P (Ei|Ek) = P (Ek|Ei)P (Ei) P (Ek)

Bayesian Methods in Positioning Applications – p.8/21

GRAZ UNIVERSITY OF TECHNOLOGY

Advanced Signal Processing Seminar

Motion model θk+1 = f(θk) + wk

θk,wk ∈ R d, f : R

d → R d

d → R mk

GRAZ UNIVERSITY OF TECHNOLOGY

Advanced Signal Processing Seminar

Likelihood function Lk(θk) = p(zk|θk) = pvk

(zk − hk(θk))

(θk+1 − f(θk))

GRAZ UNIVERSITY OF TECHNOLOGY

Advanced Signal Processing Seminar

Select initial distribution p0|0 and set k = 1

Step 1 Compute the predictive pdf pk|k−1(θk) =

∫ Rd p(θk|θk−1)p(θk−1|z1, . . . , zk−1)dθ

Step 2 Compute the posterior pdf pk|k(θk) = p(zk|θk)p(θk|z1,...,zk−1)

p(zk|z1,...,zk−1) ∝ pk|k−1(θk)Lk(θ)

Step 3 Output θk and variance V (pk|k(θk))

Step 4 Increment k and repeat from Step 1

Bayesian Methods in Positioning Applications – p.11/21

GRAZ UNIVERSITY OF TECHNOLOGY

Advanced Signal Processing Seminar

θk = argmax(pk|k(θk)) (MAP)

Problem: it is difficult to find an analytical solution to this problem!

If the noise pdfs are independent, white and zero-mean Gaussian and the measurement equation linear in θ, it can be derived that the Bayesian approach reduces to the Kalman Filter

A general algorithm which approximates the pdfs with arbitrary accuracy is the particle filter

Bayesian Methods in Positioning Applications – p.12/21

GRAZ UNIVERSITY OF TECHNOLOGY

Advanced Signal Processing Seminar

Numerical means for the tracking the evolution of the pdfs

The pdf is approximated as weighted sum of samples in the space state (particles) p(θ) ≈

∑M m=1 w(m)δ(x − x(m))

Recursive update Prediction: Calculate the predictive pdf using the

motion model Update: Weights update with normalization

Bayesian Methods in Positioning Applications – p.13/21

GRAZ UNIVERSITY OF TECHNOLOGY

Advanced Signal Processing Seminar

Particle Filter (2/4)

Initialization Select an importance function π and create a set of

particles x (m) 0 ∼ π

Measurement update Weight update according to the likelihood

w (m) k = w

(m) k−1π(θ|z)

(m) k /

∑ m w

(m) k

m=1 w (m) k θ

(m) k

Bayesian Methods in Positioning Applications – p.14/21

GRAZ UNIVERSITY OF TECHNOLOGY

Advanced Signal Processing Seminar

Weights tend to concentrate in few particles

Draw P particles from the current particle with probabilities proportional to the corresponding weight; set weights for the new particle as wk = 1

P

Optionally resampling only if a given criterion is fulfilled Neff = 1

P

GRAZ UNIVERSITY OF TECHNOLOGY

Advanced Signal Processing Seminar

GRAZ UNIVERSITY OF TECHNOLOGY

Advanced Signal Processing Seminar

1+θ2 k−1

GRAZ UNIVERSITY OF TECHNOLOGY

Advanced Signal Processing Seminar

Pro and Contra This algorithm is known under many names!

Pro: Non-linear sate model needs no approximations Non-Gaussian distributions θ

Combines different observations in a single model Variable dimension of the observation vector no problem

Contra: Computationally expensive Selection of prior distribution

Bayesian Methods in Positioning Applications – p.18/21

GRAZ UNIVERSITY OF TECHNOLOGY

Advanced Signal Processing Seminar

Application: Positioning, Navigation, Tracking

Observations: z = [pext,φins, vins,γimage,γradar,Sss, . . .]

GRAZ UNIVERSITY OF TECHNOLOGY

Advanced Signal Processing Seminar

Current research directions Motion models

Prediction of pedestrian movements Complexity reduction

Employ KF for linear states Parallelization

Distributed PF in a sensor network

Bayesian Methods in Positioning Applications – p.20/21

GRAZ UNIVERSITY OF TECHNOLOGY

Advanced Signal Processing Seminar

Recursive Identification, MIT Press, 1983.

S. Kay Fundamentals of Statistical Signal Processing, Estimation Theory, Prentice Hall PTR, 1993.

P. Djuric et al., Particle Filtering, IEEE Signal Processing Magazine, Vol.20, No.5, September 2003.

F. Gustafsson et al., Particle Filters for Positioning, Navigation and Tracking, IEEE Transactions on Signal Processing, Vol.50, Issue 2, February 2002.

N. Sirola et al., Local Positioning with Parallelepiped Moving Grid, Proceedings of The 3rd Workshop on Positioning, Navigation and Communication (WPNC), 2006.

Bayesian Methods in Positioning Applications – p.21/21

Outline

Bayes Rule

References

24. May 2006

Bayesian Methods in Positioning Applications – p.2/21

GRAZ UNIVERSITY OF TECHNOLOGY

Advanced Signal Processing Seminar

Problem Statement (1/2)

System description in two steps Find system model (modeling) Find the particular realization of the model

(identification; parameter estimation θ)

Time variant system parameters require recursive estimation on-line vs. off-line data processing

Bayesian Methods in Positioning Applications – p.3/21

GRAZ UNIVERSITY OF TECHNOLOGY

Advanced Signal Processing Seminar

With N-dimensional observation zk = {z1 k, . . . , z

N k }

Initial condition(s) - selection of θ0

Bayesian Methods in Positioning Applications – p.4/21

GRAZ UNIVERSITY OF TECHNOLOGY

Advanced Signal Processing Seminar

RLS

systems Extension of linear regression (LS) methods

Bayesian inference Estimation of distribution of θ

Bayesian Methods in Positioning Applications – p.5/21

GRAZ UNIVERSITY OF TECHNOLOGY

Advanced Signal Processing Seminar

Introduction to Bayesian Estimation

variable with p(θ)

of p(θ)

Relying on a Bayes rule updated p(θ) isinferredd from prior (predicted) p(θ) combined with incoming observations

Bayesian Methods in Positioning Applications – p.6/21

GRAZ UNIVERSITY OF TECHNOLOGY

Advanced Signal Processing Seminar

Bayes Rule and Contemporary Science

Published in T. Bayes Essay Towards Solving a Problem in the Doctrine of Chances (1764)

Steam engine of J. Watt (1769)

B. Franklin defines the concept of positive and negative polarity (1752)

L. Galvani and frogs (1770)

K.F. Gauss was born (1777)

Bayesian Methods in Positioning Applications – p.7/21

GRAZ UNIVERSITY OF TECHNOLOGY

Advanced Signal Processing Seminar

Bayes Rule

Assume a disjoint set of n discrete events {Ei} n i=1

Probabilities P (Ei) n i=1 are assigned to each event

If events are not independent observing that event Ek

hasoccurredd, probabilities of all other events {Ei} n i=1

will change

This is described using conditional probability P (Ei|Ek)

P (Ei, Ek) = P (Ei|Ek)P (Ek) = P (Ek|Ei)P (Ei)

P (Ei|Ek) = P (Ek|Ei)P (Ei) P (Ek)

Bayesian Methods in Positioning Applications – p.8/21

GRAZ UNIVERSITY OF TECHNOLOGY

Advanced Signal Processing Seminar

Motion model θk+1 = f(θk) + wk

θk,wk ∈ R d, f : R

d → R d

d → R mk

GRAZ UNIVERSITY OF TECHNOLOGY

Advanced Signal Processing Seminar

Likelihood function Lk(θk) = p(zk|θk) = pvk

(zk − hk(θk))

(θk+1 − f(θk))

GRAZ UNIVERSITY OF TECHNOLOGY

Advanced Signal Processing Seminar

Select initial distribution p0|0 and set k = 1

Step 1 Compute the predictive pdf pk|k−1(θk) =

∫ Rd p(θk|θk−1)p(θk−1|z1, . . . , zk−1)dθ

Step 2 Compute the posterior pdf pk|k(θk) = p(zk|θk)p(θk|z1,...,zk−1)

p(zk|z1,...,zk−1) ∝ pk|k−1(θk)Lk(θ)

Step 3 Output θk and variance V (pk|k(θk))

Step 4 Increment k and repeat from Step 1

Bayesian Methods in Positioning Applications – p.11/21

GRAZ UNIVERSITY OF TECHNOLOGY

Advanced Signal Processing Seminar

θk = argmax(pk|k(θk)) (MAP)

Problem: it is difficult to find an analytical solution to this problem!

If the noise pdfs are independent, white and zero-mean Gaussian and the measurement equation linear in θ, it can be derived that the Bayesian approach reduces to the Kalman Filter

A general algorithm which approximates the pdfs with arbitrary accuracy is the particle filter

Bayesian Methods in Positioning Applications – p.12/21

GRAZ UNIVERSITY OF TECHNOLOGY

Advanced Signal Processing Seminar

Numerical means for the tracking the evolution of the pdfs

The pdf is approximated as weighted sum of samples in the space state (particles) p(θ) ≈

∑M m=1 w(m)δ(x − x(m))

Recursive update Prediction: Calculate the predictive pdf using the

motion model Update: Weights update with normalization

Bayesian Methods in Positioning Applications – p.13/21

GRAZ UNIVERSITY OF TECHNOLOGY

Advanced Signal Processing Seminar

Particle Filter (2/4)

Initialization Select an importance function π and create a set of

particles x (m) 0 ∼ π

Measurement update Weight update according to the likelihood

w (m) k = w

(m) k−1π(θ|z)

(m) k /

∑ m w

(m) k

m=1 w (m) k θ

(m) k

Bayesian Methods in Positioning Applications – p.14/21

GRAZ UNIVERSITY OF TECHNOLOGY

Advanced Signal Processing Seminar

Weights tend to concentrate in few particles

Draw P particles from the current particle with probabilities proportional to the corresponding weight; set weights for the new particle as wk = 1

P

Optionally resampling only if a given criterion is fulfilled Neff = 1

P

GRAZ UNIVERSITY OF TECHNOLOGY

Advanced Signal Processing Seminar

GRAZ UNIVERSITY OF TECHNOLOGY

Advanced Signal Processing Seminar

1+θ2 k−1

GRAZ UNIVERSITY OF TECHNOLOGY

Advanced Signal Processing Seminar

Pro and Contra This algorithm is known under many names!

Pro: Non-linear sate model needs no approximations Non-Gaussian distributions θ

Combines different observations in a single model Variable dimension of the observation vector no problem

Contra: Computationally expensive Selection of prior distribution

Bayesian Methods in Positioning Applications – p.18/21

GRAZ UNIVERSITY OF TECHNOLOGY

Advanced Signal Processing Seminar

Application: Positioning, Navigation, Tracking

Observations: z = [pext,φins, vins,γimage,γradar,Sss, . . .]

GRAZ UNIVERSITY OF TECHNOLOGY

Advanced Signal Processing Seminar

Current research directions Motion models

Prediction of pedestrian movements Complexity reduction

Employ KF for linear states Parallelization

Distributed PF in a sensor network

Bayesian Methods in Positioning Applications – p.20/21

GRAZ UNIVERSITY OF TECHNOLOGY

Advanced Signal Processing Seminar

Recursive Identification, MIT Press, 1983.

S. Kay Fundamentals of Statistical Signal Processing, Estimation Theory, Prentice Hall PTR, 1993.

P. Djuric et al., Particle Filtering, IEEE Signal Processing Magazine, Vol.20, No.5, September 2003.

F. Gustafsson et al., Particle Filters for Positioning, Navigation and Tracking, IEEE Transactions on Signal Processing, Vol.50, Issue 2, February 2002.

N. Sirola et al., Local Positioning with Parallelepiped Moving Grid, Proceedings of The 3rd Workshop on Positioning, Navigation and Communication (WPNC), 2006.

Bayesian Methods in Positioning Applications – p.21/21

Outline

Bayes Rule

References