Stochastic optimization for power-aware distributed scheduling
Michael J. NeelyUniversity of Southern California
http://www-bcf.usc.edu/~mjneely
tω(t)
Outline
• Lyapunov optimization method• Power-aware wireless transmission– Basic problem– Cache-aware peering– Quality-aware video streaming
• Distributed sensor reporting and correlated scheduling
A single wireless device
R(t) = r(P(t), ω(t))
Timeslots t = {0, 1, 2, …} ω(t) = Random channel state on slot tP(t) = Power used on slot tR(t) = Transmission rate on slot t (function of P(t), ω(t))
observedchosen
Example
R(t) = log(1 + P(t)ω(t))
observedchosen
t
ω(t)
tR(t)
Example
R(t) = log(1 + P(t)ω(t))
observedchosen
t
ω(t)
tR(t)
Example
R(t) = log(1 + P(t)ω(t))
observedchosen
t
ω(t)
tR(t)
Optimization problem
Maximize: R
Subject to: P ≤ c
Given: • Pr[ω(t)=ω] = π(ω) , ω in {ω1, ω2, …, ω1000}
• p(t) in P = {p1, p2, …, p5}
• c = desired power constraint
Consider randomized decisions
Pr[pk | ωi ] = Pr[P(t) = pk | ω(t)=ωi]
• ω(t) in {ω1, ω2, …, ω1000}
• P(t) in P = {p1, p2, …, p5}
∑Pr[pk | ωi ] = 1
( for all ωi in {ω1, ω2, …, ω1000} )
k=1
5
Linear programming approach
Max: R
S.t. : P ≤ c
Given parameters: π(ωi) (1000 probabilities) r(pk , ωi) (5*1000 coefficients)
Optimization variables: Pr[pk|ωi] (5*1000 variables)
∑ ∑ π(ωi) Pr[pk|ωi] r(pk,ωi) 1000
i=1 k=1
5
∑ ∑ π(ωi) Pr[pk|ωi] pk ≤ c 1000 5
i=1 k=1
Max:
S.t.:
Multi-dimensional problem
1AccessPoint
2
N
• Observe (ω1(t), …, ωN(t))• Decisions:
-- Choose which user to serve-- Choose which power to use
R1(t)
R2(t)RN(t)
Goal and LP approach
Maximize: R1 + R2 + … + RN
Subject to: Pn ≤ c for all n in {1, …, N}
LP has given parameters: π(ω1, …, ωN) (1000N probabilities) rn(pk , ωi) (N*5N*1000N coefficients)
LP has optimization variables: Pr[pk|ωi] (5N*1000N variables)
Advantages of LP approach• Solves the problem of interest
• LPs have been around for a long time
• Many people are comfortable with LPs
Disadvantages of LP approach
Disadvantages of LP approach• Need to estimate an exponential
number of probabilities.
• LP has exponential number of variables.
• What if probabilities change?
• Fairness?
• Delay?
• Channel errors?
Lyapunov optimization approach
Maximize: R1 + R2 + … + RN
Subject to: Pn ≤ c for all n in {1, …, N}
Lyapunov optimization approach
Maximize: R1 + R2 + … + RN
Subject to: Pn ≤ c for all n in {1, …, N}
Virtual queue for each constraint:
Stabilizing virtual queue constraint satisfied!
Qn(t+1) = max[Qn(t) + Pn(t) – c, 0]
Qn(t)Pn(t) c
Lyapunov drift
L(t) = ½ ∑ Qn(t)2
Δ(t) = L(t+1) – L(t)
n
Q1
Q2
Drift-plus-penalty algorithmEvery slot t: • Observe (Q1(t), …., QN(t)), (ω1(t), …, ωN(t))
• Choose (P1(t), …, PN(t)) to greedily minimize:
• Update queues.
Δ(t) - (1/ε)(R1(t) + … + RN(t)) drift penalty
Low complexityNo knowledge of π(ω) probabilities is required
Specific DPP implementation• Each user n observes ωn(t), Qn(t). • Each user n chooses Pn(t) in P to minimize:
-(1/ε)rn(Pn(t), ωn(t)) + Qn(t)Pn(t)
• Choose user n* with smallest such value.• User n* transmits with power level Pn*(t).
Low complexityNo knowledge of π(ω) probabilities is required
Performance TheoremAssume it is possible to satisfy the constraints. Then under DPP with any ε>0: • All power constraints are satisfied.• Average thruput satisfies:
• Average queue size satisfies:
∑ Qn ≤ O(1/ε)
R1 + … + RN ≥ throughputopt – O(ε)
General SNO problem
Minimize: y0(α(t), ω(t))
Subject to: yn(α(t), ω(t)) ≤ 0 for all n in {1, …, N}
α(t) in Aω(t) for all t in {0, 1, 2, …} Such problems are solved by the DPP algorithm. Performance theorem: O(ε), O(1/ε) tradeoff.
ω(t) = Observed random event on slot t π(ω) = Pr[ω(t)=ω] (possibly unknown)α(t) = Control action on slot tAω(t) = Abstract set of action options
What we have done so far
• Lyapunov optimization method• Power-aware wireless transmission– Basic problem– Cache-aware peering– Quality-aware video streaming
• Distributed sensor reporting and correlated scheduling
What we have done so far
• Lyapunov optimization method• Power-aware wireless transmission– Basic problem– Cache-aware peering– Quality-aware video streaming
• Distributed sensor reporting and correlated scheduling
What we have done so far
• Lyapunov optimization method• Power-aware wireless transmission– Basic problem– Cache-aware peering– Quality-aware video streaming
• Distributed sensor reporting and correlated scheduling
What we have done so far
• Lyapunov optimization method• Power-aware wireless transmission– Basic problem– Cache-aware peering– Quality-aware video streaming
• Distributed sensor reporting and correlated scheduling
Mobile P2P video downloads
Mobile P2P video downloads
Mobile P2P video downloads
AccessPoint
Mobile P2P video downloads
AccessPoint
Mobile P2P video downloads
AccessPoint
Mobile P2P video downloads
AccessPoint
AccessPoint
Mobile P2P video downloads
AccessPoint
AccessPoint
AccessPoint
Mobile P2P video downloads
AccessPoint
AccessPoint
AccessPoint
Mobile P2P video downloads
AccessPoint
AccessPoint
AccessPoint
Mobile P2P video downloads
AccessPoint
AccessPoint
AccessPoint
Mobile P2P video downloads
AccessPoint
AccessPoint
AccessPoint
Mobile P2P video downloads
AccessPoint
AccessPoint
AccessPoint
Cache-aware scheduling• Access points (including “femto” nodes)• Typically stationary• Typically have many files cached
• Users• Typically mobile• Typically have fewer files cached• Assume each user wants one “long” file• Can opportunistically grab packets from
any nearby user or access point that has the file.
Quality-aware video delivery
Video chunks as time progresses
Quality Layer 1
Quality Layer 2
Bits: 8176D: 11.045
Bits: 7370D: 10.777
Quality Layer L
Bits: 40968D: 0
Bits: 58152D: 7.363
Bits: 120776D: 7.108
Bits: 97864D: 6.971
Bits: 41304D: 6.716
Bits: 277256D: 0
Bits: 419640D: 0
Bits: 72800D: 6.261
Bits: 59984D: 6.129
Bits: 299216D: 0
• D = Distortion.• Results hold for any matrices Bits(layer, chunk), D(layer, chunk).• Bits are queued for wireless transmission.
Fair video quality deliveryMinimize: f( D1 ) + f( D2 ) + … + f( DN )
Subject to: Pn ≤ c for all n in {1, …, N}
Video playback rate constraints
Fair video quality deliveryMinimize: f( D1 ) + f( D2 ) + … + f( DN )
Subject to: Pn ≤ c for all n in {1, …, N}
Video playback rate constraintsRecall the general form:
Min: y0
S.t. : yn ≤ 0 for all n α(t) in Aω(t) for all t
Fair video quality delivery
Min: y0
S.t. : yn ≤ 0 for all n α(t) in Aω(t) for all t
Minimize: f( D1 ) + f( D2 ) + … + f( DN )
Subject to: Pn ≤ c for all n in {1, …, N}
Video playback rate constraintsRecall the general form:
Define Yn(t) = Pn(t) - c
Fair video quality deliveryMinimize: f( D1 ) + f( D2 ) + … + f( DN )
Subject to: Pn ≤ c for all n in {1, …, N}
Video playback rate constraintsRecall the general form:
Define auxiliary variable γ(t) in [0, Dmax]
Min: y0
S.t. : yn ≤ 0 for all n α(t) in Aω(t) for all t
Equivalence via Jensen’s inequalityMinimize: f( D1 ) + f( D2 ) + … + f( DN )
Subject to: Pn ≤ c for all n in {1, …, N}
Video playback rate constraints
Minimize: f( γ1(t)) + f( γ2(t)) + … + f( γN(t))
Subject to: Pn ≤ c for all n in {1, …, N}
γn = Dn for all n in {1, …, N} Video playback rate constraints
Example simulation
BS
• Region divided into 20 x 20 subcells (only a portion shown here).• 1250 mobile devices, 1 base station • 3.125 mobiles/subcell
• Phases 1, 2, 3: File availability prob = 5%, 10%, 7%• Basestation Average Traffic: 2.0 packets/slot• Peer-to-Peer Average Traffic: 153.7 packets/slot• Factor of 77.8 gain compared to BS alone!
What we have done so far
• Lyapunov optimization method• Power-aware wireless transmission– Basic problem– Cache-aware peering– Quality-aware video streaming
• Distributed sensor reporting and correlated scheduling
Distributed sensor reports
• ωi(t) = 0/1 if sensor i observes the event on slot t• Pi(t) = 0/1 if sensor i reports on slot t• Utility: U(t) = min[P1(t)ω1(t) + (1/2)P2(t)ω2(t),1]
1
2Fusion Center
Maximize: U
Subject to: P1 ≤ c
P2 ≤ c
ω1(t)
ω2(t)
What is optimal?
Agreeon plan 0 1 2 3
t4
What is optimal?
Agreeon plan 0 1 2 3
t4
Example plan: User 1: • t=even Do not report.• t=odd Report if ω1(t)=1.User 2: • t=even Report if ω2(t)=1 • t=odd: Report with prob ½ if ω2(t)=1
Common source of randomness
Example: 1 slot = 1 dayEach user looks at Boston Globe every day:• If first letter is a “T” Plan 1• If first letter is an “S” Plan 2• Etc.
Day 1 Day 2
Specific exampleAssume:• Pr[ω1(t)=1] = ¾, Pr[ω2(t)=1] = ½• ω1(t), ω2(t) independent• Power constraint c = 1/3
Approach 1: Independent reporting• If ω1(t)=1, user 1 reports with probability θ1
• If ω2(t)=1, user 2 reports with probability θ2
Optimizing θ1, θ2 gives u = 4/9 ≈ 0.44444
Approach 2: Correlated reportingPure strategy 1: • User 1 reports if and only if ω1(t)=1.• User 2 does not report.
Pure strategy 2: • User 1 does not report.• User 2 reports if and only if ω2(t)=1.
Pure strategy 3:• User 1 reports if and only if ω1(t)=1.• User 2 reports if and only if ω2(t)=1.
Approach 2: Correlated reportingX(t) = iid random variable (commonly known): • Pr[X(t)=1] = θ1
• Pr[X(t)=2] = θ2
• Pr[X(t)=3] = θ3
On slot t: • Users observe X(t)• If X(t)=k, users use pure strategy k.
Optimizing θ1, θ2, θ3 gives u = 23/48 ≈ 0.47917
Summary of approaches
Independent reporting
Correlated reporting
Centralized reporting
0.47917
0.44444
0.5
Strategy u
Summary of approaches
Independent reporting
Correlated reporting
Centralized reporting
0.47917
0.44444
0.5
Strategy u
It can be shown that this is optimal over all distributed strategies!
General distributed optimization
Maximize: U
Subject to: Pk ≤ 0 for k in {1, …, K}
ω(t) = (ω1(t), …, ωΝ(t))π(ω) = Pr[ω(t) = (ω1, …, ωΝ)]α(t) = (α1(t), …, αΝ(t))U(t) = u(α(t), ω(t))Pk(t) = pk(α(t), ω(t))
Pure strategies
A pure strategy is a deterministic vector-valued function:
g(ω) = (g1(ω1), g2(ω2), …, gΝ (ωΝ))
Let M = # pure strategies:
M = |A1||Ω1| x |A2||Ω2| x ... x |AN||ΩN|
Optimality Theorem
There exist:• K+1 pure strategies g(m)(ω)• Probabilities θ1, θ2, …, θK+1
such that the following distributed algorithm is optimal:
X(t) = iid, Pr[X(t)=m] = θm
• Each user observes X(t)• If X(t)=m use strategy g(m)(ω).
LP and complexity reduction• The probabilities can be found by an LP
• Unfortunately, the LP has M variables
• If (ω1(t), …, ωΝ(t)) are mutually independent and the utility function satisfies a preferred action property, complexity can be reduced
• Example N=2 users, |A1|=|A2|=2
--Old complexity = 2|Ω1|+|Ω2|
--New complexity = (|Ω1|+1)(|Ω2|+1)
Lyapunov optimization approach• Define K virtual queues Q1(t), …, QK(t).
• Every slot t, observe queues and choose strategy m in {1, …, M} to maximize a weighted sum of queues.
• Update queues with delayed feedback:
Qk(t+1) = max[Qk(t) + Pk(t-D), 0]
Separable problemsIf the utility and penalty functions are a separable sum of functions of individual variables (αn(t), ωn(t)), then:
• There is no optimality gap between centralized and distributed algorithms
• Problem complexity reduces from exponential to linear.
Simulation (non-separable problem)
• 3-user problem• αn(t) in {0, 1} for n ={1, 2, 3}. • ωn(t) in {0, 1, 2, 3, 4, 5, 6, 7, 8, 9}• V=1/ε• Get O(ε) guarantee to optimality• Convergence time depends on 1/ε
Utility versus V parameter (V=1/ε)U
tility
V (recall V = 1/ε)
Average power versus timeAv
erag
e po
wer
up
to ti
me
t
Time t
power constraint 1/3V=10
V=50
V=100
Adaptation to non-ergodic changes
Conclusions
• Drift-plus-penalty is a strong technique for general stochastic network optimization
• Power-aware scheduling
• Cache-aware scheduling
• Quality-aware video streaming
• Correlated scheduling for distributed stochastic optimization
Conclusions
• Drift-plus-penalty is a strong technique for general stochastic network optimization
• Power-aware scheduling
• Cache-aware scheduling
• Quality-aware video streaming
• Correlated scheduling for distributed stochastic optimization
Top Related