Distributed Stochastic Optimization via Correlated Scheduling

Michael J. NeelyUniversity of Southern California

http://www-bcf.usc.edu/~mjneely

1

2

Fusion Center

Observation ω1(t)

Observation ω2(t)

1

Distributed sensor reports

• ωi(t) = 0/1 if sensor i observes the event on slot t• Pi(t) = 0/1 if sensor i reports on slot t• Utility: U(t) = min[P1(t)ω1(t) + (1/2)P2(t)ω2(t),1]

1

2Fusion Center

ω1(t)

ω2(t)

2

Redundant reports do not increase utility.

Distributed sensor reports

• ωi(t) = 0/1 if sensor i observes the event on slot t• Pi(t) = 0/1 if sensor i reports on slot t• Utility: U(t) = min[P1(t)ω1(t) + (1/2)P2(t)ω2(t),1]

1

2Fusion Center

Maximize: U

Subject to: P1 ≤ c

P2 ≤ c

ω1(t)

ω2(t)

3

Main ideas for this example4

• Utility function is non-separable.• Redundant reports do not bring extra utility.• A centralized algorithm would never send

redundant reports (it wastes power).• A distributed algorithm faces these

challenges: • Sensor 2 does not know if sensor 1

observed an event.• Sensor 2 does not know if sensor 1

reported anything.

Assumed structure

Agreeon plan 0 1 2 3

t4

5

1) Coordinate on a plan before time 0.

2) Distributively implement plan after time 0.

Example “plans”

Agreeon plan 0 1 2 3

t4

Example plan: Sensor 1: • t=even Do not report.• t=odd Report if ω1(t)=1.Sensor 2: • t=even Report with prob p if ω2(t)=1 • t=odd: Do not report.

6

Common source of randomness

Example: 1 slot = 1 dayEach person looks at Boston Globe every day:• If first letter is a “T” Plan 1• If first letter is an “S” Plan 2• Etc.

Day 1 Day 2

7

Specific exampleAssume:• Pr[ω1(t)=1] = ¾, Pr[ω2(t)=1] = ½• ω1(t), ω2(t) independent• Power constraint c = 1/3

Approach 1: Independent reporting• If ω1(t)=1, sensor 1 reports with probability θ1

• If ω2(t)=1, sensor 2 reports with probability θ2

Optimizing θ1, θ2 gives u = 4/9 ≈ 0.44444

8

Approach 2: Correlated reportingPure strategy 1: • Sensor 1 reports if and only if ω1(t)=1.• Sensor 2 does not report.

Pure strategy 2: • Sensor 1 does not report.• Sensor 2 reports if and only if ω2(t)=1.

Pure strategy 3:• Sensor 1 reports if and only if ω1(t)=1.• Sensor 2 reports if and only if ω2(t)=1.

9

Approach 2: Correlated reportingX(t) = iid random variable (commonly known): • Pr[X(t)=1] = θ1

• Pr[X(t)=2] = θ2

• Pr[X(t)=3] = θ3

On slot t: • Sensors observe X(t)• If X(t)=k, sensors use pure strategy k.

Optimizing θ1, θ2, θ3 gives u = 23/48 ≈ 0.47917

10

Summary of approaches

Independent reporting

Correlated reporting

Centralized reporting

0.47917

0.44444

0.5

Strategy u

11

Summary of approaches

Independent reporting

Correlated reporting

Centralized reporting

0.47917

0.44444

0.5

Strategy u

It can be shown that this is optimal over all distributed strategies!

12

General distributed optimization

Maximize: U

Subject to: Pk ≤ c for k in {1, …, K}

ω(t) = (ω1(t), …, ωΝ(t))π(ω) = Pr[ω(t) = (ω1, …, ωΝ)]α(t) = (α1(t), …, αΝ(t))U(t) = u(α(t), ω(t))Pk(t) = pk(α(t), ω(t))

13

Pure strategies

A pure strategy is a deterministic vector-valued function:

g(ω) = (g1(ω1), g2(ω2), …, gΝ (ωΝ))

Let M = # pure strategies:

M = |A1||Ω1| x |A2||Ω2| x ... x |AN||ΩN|

14

Optimality Theorem

There exist:• K+1 pure strategies g(m)(ω)• Probabilities θ1, θ2, …, θK+1

such that the following distributed algorithm is optimal:

X(t) = iid, Pr[X(t)=m] = θm

• Each user observes X(t)• If X(t)=m use strategy g(m)(ω).

15

LP and complexity reduction• The probabilities can be found by an LP

• Unfortunately, the LP has M variables

• If (ω1(t), …, ωΝ(t)) are mutually independent and the utility function satisfies a preferred action property, complexity can be reduced

• Example N=2 users, |A1|=|A2|=2

--Old complexity = 2|Ω1|+|Ω2|

--New complexity = (|Ω1|+1)(|Ω2|+1)

16

Discussion of Theorem 1Theorem 1 solves the problem for distributed scheduling, but:• Requires an offline LP to be solved before

time 0.

• Requires full knowledge of π(ω) probabilities.

17

Online Dynamic ApproachWe want an algorithm that: • Operates online• Does not need π(ω) probabilities. • Can adapt when these probabilities

change.

18

Such an algorithm must use feedback: • Assume feedback is a fixed delay D.• Assume D>1. • Such feedback cannot improve average

utility beyond the distributed optimum.

Lyapunov optimization approach• Define K virtual queues Q1(t), …, QK(t).

• Every slot t, observe queues and choose strategy m in {1, …, M} to maximize a weighted sum of queues.

• Update queues with delayed feedback:

Qk(t+1) = max[Qk(t) + Pk(t-D) - c, 0]

19

Lyapunov optimization approach• Define K virtual queues Q1(t), …, QK(t).

• Every slot t, observe queues and choose strategy m in {1, …, M} to maximize a weighted sum of queues.

• Update queues with delayed feedback:

Qk(t+1) = max[Qk(t) + Pk(t-D) - c, 0]

20

Virtual queue: If stable, then: Time average power ≤ c.

“arrivals” “service”

Separable problemsIf the utility and penalty functions are a separable sum of functions of individual variables (αn(t), ωn(t)), then:

• There is no optimality gap between centralized and distributed algorithms

• Problem complexity reduces from exponential to linear.

21

Simulation (non-separable problem)

• 3-user problem• αn(t) in {0, 1} for n ={1, 2, 3}. • ωn(t) in {0, 1, 2, 3, 4, 5, 6, 7, 8, 9}• V=1/ε• Get O(ε) guarantee to optimality• Convergence time depends on 1/ε

22

Utility versus V parameter (V=1/ε)U

tility

V (recall V = 1/ε)

23

Average power versus timeAv

erag

e po

wer

up

to ti

me

t

Time t

power constraint 1/3V=10

V=50

V=100

24

Adaptation to non-ergodic changes25

Adaptation to non-ergodic changes26

Optimal utility for phase 2

Optimal utility for phases 1 and 3

Oscillates about the average constraint c

Conclusions• Paper introduces correlated scheduling via

common source of randomness.• Common source of randomness is crucial for

optimality in a distributed setting. • Optimality gap between distributed and

centralized problems (gap=0 for separable problems).

• Complexity reduction technique in paper.• Online implementation via Lyapunov optimization.• Online algorithm adapts to a changing

environment.

27