Distributedconsensus-basedBayesian estimation...

Distributed consensus-based Bayesianestimation: sufficient conditions for performance

characterization

Damiano Varagnolo, Gianluigi Pillonetto, Luca Schenato

Department of Information Engineering - University of Padova

ACC - July 1st , 2010

entirely

writtenin

LATEX2ε using

Beamer and Tik Z

varagnolo pillonetto schenato (Padova) damiano.varagnolo@dei.unipd.it ACC - July 1st , 2010 1 / 21

Networked Control Systems Group - Padova, Italy

SaverioBolognani

AngeloCenedese

AlessandroChiuso

EnricoLovisari

GianluigiPillonetto

LucaSchenato

DamianoVaragnolo

SandroZampieri

Introductive Mindmap

This Talkintroduction

frame-work

moti-vatingexample

measure-mentmodel

estimators

cen-tralized

dis-tributed

bounds

differentsensors

samesensors

Introduction

Framework

Distributed Estimationmeasurements from many sensors

no central coordinating unit

no direct communications

limited sensors knowledge

no time dependency (no dynamic systems)

Distributed Algorithms should. . .do not rely on a-priori topology knowledge

be robust to node failure and topology changes

Motivations

A Motivating Example(artificially built on ZebraNet project)

Network =

{bunch of zebraswith wireless sensors

Want to relate:

temperature

humidity

thirstyness

(image: thundafunda.com)

We can use. . .either “local estimates” (bad since inexpensive sensors)

or try to exploit sensors redundancyvaragnolo pillonetto schenato (Padova) damiano.varagnolo@dei.unipd.it ACC - July 1st , 2010 5 / 21

Contributions

Our contributions

Existing literature on:bounds on the number of sensors / measurements to obtain adesired level of accuracy (ex. [Kearns and Seung, 1995],[Yamanishi, 1997])

Our focus on:fully distributed algorithms

comparisons between local and distributed algorithms

Main contribution: sufficient conditions assuring “to share is better”

Problem statement

Measurement Model

yi = Cia + νi , i = 1, . . . , S (1)

where:M : number of measurementsS : number of sensorsP : number of parameters (M � P)

yi ∈ RM : vector of measurements from sensor ia ∈ RP : vector of unknown parameters s.t. a ∼ N (0,Σa)

νi ∈ RM : vector of noises s.t. νi ∼ N (0, σ2i IM) (i.i.d.)

will focus on the case Ci = C

Problem statement

Standard Bayesian estimatorsOriginal formulation

Local estimator:

aloc,i := cov (a, yi) var (yi)−1 yi (2)

Centralized estimator:

acent := cov

y1...yS

−1 y1

Problem statement

Standard Bayesian estimatorsNumerical formulation

Local estimator:

aloc,i := ΣaCT (CΣaCT + σ2i IM)−1 yi (4)

Centralized estimator:

acent := ΣaCT

CΣaCT +

−1 ∑Si=1

i∑Si=1

Problem statement

Distributing the Centralized Bayesian estimators

Definition: harmonic mean of the measurements noises variances:

h :=S∑S

i=11σ2

∑Si=1

→ average consensus (6)

If all sensors know S then centralized estimator can be computingusing average consensus:

acent :=

S∑i=1

ΣaCT(

CΣaCT +hS· IM)−1 yi

S∑i=1

Distributed Estimator

. . .and if the sensors do not know the number ofsensors S?

use a guess S : adist(S)

S∑i=1

ΣaCT(

CΣaCT +hS· IM)−1 yi

S∑i=1

(8)For sure behaves not better w.r.t. the centralized estimator:

var(adist

(S)− a)≥ var (acent − a) ∀ Σa , h, C , P , M .

And w.r.t. the local one??varagnolo pillonetto schenato (Padova) damiano.varagnolo@dei.unipd.it ACC - July 1st , 2010 11 / 21

First bound: when a single sensor is sure toperform better with the distributed strategy

[S −

√S2 − Sh

σ2i, S +

√S2 − Sh

var(adist

(S)− a)< var (aloc,i − a) ∀ Σa , h, C , P , M .

Second bound: when all the sensors are sure toperform better with the distributed strategy

Define σ2min := mini {σ2

i }. If:

[S −

√S2 − Sh

σ2min, S +

√S2 − Sh

σ2min

var(adist

(S)− a)< var (aloc,i − a) ∀ i , Σa , h, C , P , M .

Third bound: when sensors “in average” performbetter with the distributed strategy

If:S ∈

[S −√

S2 − S , S +√

S2 − S]

var(adist

(S)− a)<

S∑i=1

var (aloc,i − a) ∀ Σa , h, C , P , M .

Graphical intuition of the bounds

S1 2 3 S

first bound: given sensor performs better with given strategy

[S −

√S2 − Sh

σ2i, S +

√S2 − Sh

[S −

√S2 − Sh

σ2i, S +

√S2 − Sh

1) noise increases ⇒ bound grows

example: σ2i = 10

[S −

√S2 − Sh

σ2i, S +

√S2 − Sh

example: σ2i = 20

[S −

√S2 − Sh

σ2i, S +

√S2 − Sh

example: σ2i = 30

[S −

√S2 − Sh

σ2i, S +

√S2 − Sh

2) S increases ⇒ bound shifts and grows

example: S = 10

[S −

√S2 − Sh

σ2i, S +

√S2 − Sh

example: S = 20

[S −

√S2 − Sh

σ2i, S +

√S2 − Sh

example: S = 30

second and third bounds: similar behaviors

• noisiness increases ⇒ bounds grow

• S increases ⇒ bounds grow and shift

relations between the bounds:

better for all (2nd)

better in average (3th)

better in average (3th)better for sensor i (1st)

Simplified situationWhen all sensors are evenly noisy: σ2

i = σ2

Optimal centralized estimator has simplified structure:

acent :=1S

S∑i=1

ΣaCT(

CΣaCT +σ2

yi (12)

still requires the knowledge of S!

If not, use a guess S :

adist(S)

S∑i=1

ΣaCT(

CΣaCT +σ2

yi (13)

Bounds for the simplified situation

How the bounds on S s.t.:

var(adist

(S)− a)< var (aloc,i − a) ∀ Σa , α, C , P , M

modify??

directly previous results:

S ∈[S −√

S2 − S , S +√

S2 − S]

using different proofs:

S ∈ [1, 2 (S − 1)] (15)

An useful corollary

S ∈ [1, 2 (S − 1)] ⇒ can choose S = 1

that imply:aloc,i = ΣaCT (CΣaCT + σ2IM

)−1 yi (16)

is worse than:

adist (1) =1S

S∑i=1

ΣaCT (CΣaCT + σ2IM)−1 yi =

S∑i=1

aloc,i (17)

i.e. always better to share local optimal estimates (once computed)!

Conclusions and Future Works

Conclusionsthere exists bounds assuring distributed estimators to behave“better” than local ones (under mild assumptions)

can use these bounds to justify naïve algorithms

Future Worksinstead of C consider sensor dependent Ci

consider non-parametric function estimation (infinite-dimensionalfunctions instead of finite-dimensional vectors)

Distributed consensus-based Bayesianestimation: sufficient conditions for performance

characterization

Damiano Varagnolo, Gianluigi Pillonetto, Luca Schenato

Department of Information Engineering - University of Padova

ACC - July 1st , 2010

damiano.varagnolo@dei.unipd.itwww.dei.unipd.it/∼varagnolo/google: damiano varagnoloen

tirely

writtenin

LATEX2ε using

Beamer and Tik Z

Example of how average consensus works[Cortés, 2008]

a:b:c:d:

20.513.519.618.0

a:b:c:d:

17.017.019.618.0

a:b:c:d:

18.317.018.318.0

a:b:c:d:

b t = 4

18.317.018.1518.15

a:b:c:d:

18.317.57518.1517.575

a:b:c:d:

17.937517.57518.1517.9375

a:b:c:d:

17.937517.862517.862517.9375

a:b:c:d:

17.917.862517.9

17.9375

a:b:c:d:

b t = 9

17.917.862517.918717.9187

Cortés, J. (2008).Automatica .

Kearns, M. and Seung, H. S. (1995).Machine Learning 18 (2-3), 255–276.

Yamanishi, K. (1997).In: COLT ’97: Proceedings of the tenth annual conference onComputational learning theory pp. 250–262, New York, NY,USA: ACM.

Distributedconsensus-basedBayesian estimation...

Documents

Transcript of Distributedconsensus-basedBayesian estimation...

K i t t P e ak N i gh t l y O b s e r vi n g P rogr am · PDF file04.04.2017 · Tr i a ngl e a s t e r i s m . I t s l a rge vol um e s ugge s t s t ha t i f i t w e r

Population Review. Exponential growth N t+1 = N t + B – D + I – E ΔN = B – D + I – E For a closed population ΔN = B – D.

a l 2014 I n s t i t DIRECTOR’S u t e a REPORT t A t h e n ssydney.edu.au/arts/aaia/docs/publications... · Dr Paul Donnelly – The ... The Estate of the late Professor J.A. Young

N Σύνεδροι Ω I N A Ω I Y O I M H T Σ I Π NE A Π · Ανθρωπολογία» 17.30-17.45 Λήδα Παπαστεφανάκη,Εισαγωγή στην προβληματική

Kinetic Monte Carlo Methods...Brian Munsky C e n t e r f o r C o n t r o l, D y n a m i c a l S y s t e m s Simulation Methods a n d C o m p u t a i o n Kinetic Monte-Carlo CC DC•

π Δ g o k n o ^ μ a f g o x t n. K f ^ π ^ i Y h n o k Twitterusers.uoa.gr/~gmikros/Pdf/The Cypriot dialect in social media.pdf · π p Y l f eμ ( g k f i t i f k) h t n n k

Solution to selected problems. - Columbia Universityprotter/sde_solution_chapter123.pdf · 2011. 4. 8. · 10. (a) Let Nt = P i 1 i (N i t ¡ t) and Lt = P i 1 i (L i t ¡ t).As we

1-OK COVER-JULY - chulabhornhospital.com · 4 P a t i e n t F i r s t Chulabhorn Hospital I ISSUE 05 I April-June 2017: Inside Update «‘™“™ ’æ√ —ß ’‡∑§π ‘§À√

TEORI CAHAYA - sugito.staff.telkomuniversity.ac.idsugito.staff.telkomuniversity.ac.id/files/2017/01/BTEORICAHAYA.pdf · Fakultas Teknik Elektro Teori Cahaya n 1 n 2 i r t i = r 2

C o m b i n at o r i c s an d S t at i s t i c al P h y s i c …math.mit.edu/conferences/wim/womeninmath/conference/...¥ T h e m o d e l is a s y m m e t r i c in t h e s e n s e

Simple Linear Regression - Cengage · PDF files t a t i s t i c s f o r b u s i n e s s a n d e o n o m i c simple linear regression contents statistics in practice: alliance data

T O P L O T N I T R A N S F E R - GAFgaf.ni.ac.rs/fizika/doc/Zbirka zadataka/toplotni transfer...16 T O P L O T N I T R A N S F E R Provo|enje toplote 1. Me{avina leda i vode nalazi

P R O F E S S I O N A L E X A M I N A T I O N B O A R Dpeb.mp.gov.in/results/RESULT_17/ABV_RES17/ABV_2017_FINAL...14) 15) 16) 3. P ro t e ins / 埉 ट न4. N o ne o f t he s e

O F F I C I A L C O N T E S T R E S U L T S - IFBB · PDF fileO F F I C I A L C O N T E S T R E S U L T SKong R1 R2 R3 R4 1 Tu Deng Yu-Hsi Chinese Taipei /157 PLACE 2 Lin Qin Yu China

P R O D U K T I N F O R M A T I O N...P R O D U K T I N F O R M A T I O N Rohrdetektorsystem auf Plastikszintillatorbasis Zur Messung und Kontrolle von Kontaminationen in Innenrohren

I O A N N I S K O N S T A N T A R A S 1. Personal Informationusers.uom.gr/~ikonst/cvenglish.pdf · K Skouri, S. Papachristos, I. Konstantaras and I. Ganas, An inventory model for

E RS T R I N G CHEES E I T H KS N C O N T H I D04 CARRY ON WAYWARD SON > 6:34 05 ON THE ROAD 5:16 ENCORE 06 CHATTER 1:59 07 MIDNIGHT MOONLIGHT 7:54 08 sIT IS WHAT IT IS a12:37. U n

E N T I T É S S T A T I Q U E S EP

Brownian Bridgepitman/s205s03/b_bridge.pdf · 22-2 Brownian Bridge And EHn(t) = 0; VarHn(t) = t(1¡t) E(Hn(s)Hn(t)) = Cov( 1 p n Xn i=1 1(Ui • s);1 p n Xn j=1 1(Uj • t))1 n Xn

14. Total Internal Reflection and Evanescent Waves...Total Internal Reflection n i n t k i k t θ i θ t E i E t Interface Snell’s Law: sin sinn n i i t tθ θ= Solve for θ t :