1

TCOM 501: Networking Theory & Fundamentals

Lectures 4 & 5February 5 and 12, 2003

Prof. Yannis A. Korilis

4&5-2 Topics Markov Chains M/M/1 Queue Poisson Arrivals See Time Averages M/M/* Queues Introduction to Sojourn Times

4&5-3 The M/M/1 Queue Arrival process: Poisson with rate λ Service times: iid, exponential with parameter

µ Service times and interarrival times:

independent Single server Infinite waiting room N(t): Number of customers in system at time t

(state)0 1 n+1n2

4&5-4 Exponential Random Variables X: exponential RV with

parameter λ Y: exponential RV with

parameter μ X, Y: independentThen:1. min{X, Y}: exponential RV

with parameter λ+μ2. P{X<Y} = λ/(λ+μ)

[Exercise 3.12]

Proof:

0 0

0 0

0 0

0

( )

0 0

{ } ( , )

(1 )

( )

1

y

XY

yx y

yy x

y y

y y

P X Y f x y dx dy

e e dx dy

e e dx dy

e e dy

e dy e dy

( )

( )

{min{ , } } { , }{ } { }

{min{ , } } 1

t t t

t

P X Y t P X t Y tP X t P Y te e e

P X Y t e

4&5-5

M/M/1 Queue: Markov Chain Formulation

Jumps of {N(t): t ≥ 0} triggered by arrivals and departures{N(t): t ≥ 0} can jump only between neighboring states

Assume process at time t is in state i: N(t) = i ≥ 1 Xi: time until the next arrival – exponential with parameter λ Yi: time until the next departure – exponential with

parameter μ Ti =min{Xi,Yi}: time process spends at state i Ti : exponential with parameter νi= λ+μ Pi,i+1=P{Xi < Yi}= λ/(λ+μ), Pi,i-1=P{Yi < Xi}= μ/(λ+μ)

P01=1, and T0 is exponential with parameter λ

{N(t): t ≥ 0} is a continuous-time Markov chain with, 1 , 1

, 1 , 1

, 0, 1

0, | | 1

i i i i i

i i i i i

ij

q P iq P iq i j

4&5-6

M/M/1 Queue: Stationary Distribution

Birth-death process → DBE

Normalization constant

Stationary distribution

0 1 n+1n2

1

1 1 0...

n n

nn n n

p p

p p p p

0 00 1

1 1 1 1 , if 1nn

n n

p p p

(1 ), 0,1,...nnp n

4&5-7

The M/M/1 Queue

Average number of customers in system

Little’s Theorem: average time in system

Average waiting time and number of customers in the queue – excluding service

1

0 0 0

2

(1 ) (1 )

1(1 )

(1 ) 1

n nn

n n n

N np n n

N

11NT

1 and

1 2

WNTW Q

4&5-8 The M/M/1 Queue

ρ=λ/μ: utilization factor Long term proportion of

time that server is busy ρ=1-p0: holds for any

M/G/1 queue Stability condition: ρ<1

Arrival rate should be less than the service rate

0

2

4

6

8

10

0 0.2 0.4 0.6 0.8 1

N

4&5-9

M/M/1 Queue: Discrete-Time Approach

Focus on times 0, , 2,… ( arbitrarily small) Study discrete time process Nk = N(k)

Show that transition probabilities are

Discrete time Markov chain, omitting o(

lim { ( ) } lim { }kt k

P N t n P N n

00

, 1

, 1

1 ( )

1 ( ), 1

( ), 0

( ), 0

( ), | | 1

ii

i i

i i

ij

P

P i

P i

P i

P i j

0 1 n+1n2

1 1 1 1

4&5-10

M/M/1 Queue: Discrete-Time Approach

Discrete-time birth-death process → DBE:

Taking the limit →0:

Done!

1

1 0

[ ( )]π [ ( )]π

( ) ( )π π π

( ) ( )

n n

n

n n

0 00 0 0

( )lim π lim lim π

( )

n n

n np p

0 1 n+1n2

1 1 1 1

4&5-11 Transition Probabilities? Ak: number of customers that arrive in Ik=(k, (k+1)]

Dk: number of customers that depart in Ik=(k, (k+1)] Transition probabilities Pij depend on conditional

probabilities: Q(a,d | n) = P{Ak=a, Dk=d | Nk-1=n} Calculate Q(a,d | n) using arrival and departure statistics Use Taylor expansion e-=1-+o(), e-=1-+o(), to

express as a function of Poisson arrivals: P{Ak ≥ 2}=o( Probability there are more than 1 arrivals in Ik is o() Show: probability of more than one event (arrival or

departure) in Ik is o() See details in textbook

4&5-12 Example: Slowing Down

M/M/1 system: slow down the arrival and service rates by the same factor m

Utilization factors are the same ⇒stationary distributions the same, average number in the system the same

Delay in the slower system is m times higher Average number in queue is the same, but in the 1st

system the customers move out faster

/

1 1 /

/

( / )

/ /

N N

N mT mT

m

mW mW

m m

/ m/ m

/

1 1 /

1

/

N

NT

W

4&5-13

Example: Statistical MUX-ing vs. TDM

m identical Poisson streams with rate λ/m; link with capacity 1; packet lengths iid, exponential with mean 1/μ

Alternative: split the link to m channels with capacity 1/m each, and dedicate one channel to each traffic stream

Delay in each “queue” becomes m times higher Statistical multiplexing vs. TDM or FDMWhen is TDM or FDM preferred over statistical multiplexing?

mT mT

/ m/ m

/ m/ m

m

4&5-14 “PASTA” Theorem Markov chain: “stationary” or “in steady-state:”

Process started at the stationary distribution, or Process runs for an infinite time t→∞

Probability that at any time t, process is in state i is equal to the stationary probability

Question: For an M/M/1 queue: given t is an arrival time, what is the probability that N(t)=i?Answer: Poisson Arrivals See Time Averages!

( )lim { ( ) } lim i

it t

T tp P N t i

t

4&5-15 PASTA Theorem Steady-state probabilities:

Steady-state probabilities upon arrival:

Lack of Anticipation Assumption (LAA): Future inter-arrival times and service times of previously arrived customers are independent

Theorem: In a queueing system satisfying LAA:1. If the arrival process is Poisson:

2. Poisson is the only process with this property (necessary and sufficient condition)

lim { ( ) }nt

p P N t n

lim { ( ) | arrival at }nt

a P N nt t

, 0,1,...n na p n

4&5-16 PASTA Theorem

Doesn’t PASTA apply for all arrival processes? Deterministic arrivals every 10 sec Deterministic service times 9 sec

Upon arrival: system is always empty a1=0

Average time with one customer in system: p1=0.9

“Customer” averages need not be time averages Randomization does not help, unless Poisson!

10 109 20 30 30 39

4&5-17 PASTA Theorem: Proof Define A(t,t+δ), the event that an arrival occurs in [t, t+

δ) Given that a customer arrives at t, probability of finding

the system in state n:

A(t,t+δ) is independent of the state before time t, N(t-) N(t-) determined by arrival times <t, and corresponding

service times A(t,t+δ) independent of arrivals <t [Poisson] A(t,t+δ) independent of service times of customers arrived

<t [LAA]0 0

0

{ ( ) , ( , )}( ) lim { ( ) | ( , )} lim

{ ( , )}

{ ( ) } { ( , )}lim { ( ) }

{ ( , )}

n

P N t n A t ta t P N t n A t t

P A t t

P N t n P A t tP N t n

P A t t

lim ( ) lim { ( ) }n n nt ta a t P N t n p

0{ ( ) | arrival at t} lim { ( ) | ( , )}P N t n P N t n A t t

4&5-18 PASTA Theorem: Intuitive Proof ta and tr: randomly selected arrival and observation

times, respectively The arrival processes prior to ta and tr respectively are

stochastically identical The probability distributions of the time to the first arrival

before ta and tr are both exponentially distributed with parameter

Extending this to the 2nd, 3rd, etc. arrivals before ta and tr establishes the result

State of the system at a given time t depends only on the arrivals (and associated service times) before t Since the arrival processes before arrival times and random times are identical, so is the state of the system they see

4&5-19

Arrivals that Do not See Time-Averages

Example 1: Non-Poisson arrivals IID inter-arrival times, uniformly distributed between in 2

and 4 sec Service times deterministic 1 sec

Upon arrival: system is always emptyλ=1/3, T=1 → N=T/λ=1/3 → p1=1/3

Example 2: LAA violated Poisson arrivals Service time of customer i: Si= Ti+1, 1

Upon arrival: system is always emptyAverage time the system has 1 customer: p1=

4&5-20 Distribution after Departure Steady-state probabilities after departure:

Under very general assumptions: N(t) changes in unit increments

limits an and exist dn

an = dn, n=0,1,…

In steady-state, system appears stochastically identical to an arriving and departing customer

Poisson arrivals + LAA: an arriving and a departing customer see a system that is stochastically to the one seen by an observer looking at an arbitrary time

| departure at lim { ( ) }nt

d P X nt t

4&5-21 M/M/* Queues Poisson arrival process

Interarrival times: iid, exponential Service times: iid, exponential Service times and interarrival times: independent N(t): Number of customers in system at time t

(state){N(t): t ≥ 0} can be modeled as a continuous-time Markov chainTransition rates depend on the characteristics of the systemPASTA Theorem always holds

4&5-22 M/M/1/K Queue

M/M/1 with finite waiting room At most K customers in the system Customer that upon arrival finds K customers in system is

dropped Stationary distribution

Stability condition: always stable – even if ρ≥1 Probability of loss – using PASTA theorem:

0 1 KK-12

0

0 1

, 1,2,...,1

1

nn

K

p p n K

p

1

(1 ){loss} { ( ) }

1

K

KP P N t K

4&5-23 M/M/1/K Queue (proof)

Exactly as in the M/M/1 queue:

Normalization constant:

Generalize: Truncating a Markov chain

0 1 KK-12

1

0 00 1

0 1

11 1 1

11

1

KK Kn

nn n

K

p p p

p

0 , 1,2,...,nnp p n K

4&5-24 Truncating a Markov Chain

{X(t): t ≥ 0} continuous-time Markov chain with stationary distribution {pi: i=0,1,…}

S a subset of {0,1,…}: set of states; Observe process only in S

Eliminate all states not in S Set

{Y(t): t ≥ 0}: resulting truncated process; If irreducible: Continuous-time Markov chain Stationary distribution

Under certain conditions – need to verify depending on the system

if

0 if

j

ij

i S

pj S

pp

j S

0, ,ji ijq q j S i S

4&5-25

Truncating a Markov Chain (cont.) Possible sufficient condition

Verify that distribution of truncated process1. Satisfies the GBE

2. Satisfies the probability conservation law:

Another – even better – sufficient condition: DBE!Relates to “reversibility”Holds for multidimensional chains

( ) ( ),

j ij ji i ij j ji i ij ji ij

i i i S i S i S i S

j ji i ij j ji i iji S i S i S i S

p pp q p q p q p q q q

p S p Sp q p q p q p q j S

,j ji i iji S i S

p q p q j S

( )1, ( )

( ) ( )i

i ii S i S i S

p p Sp p S p

p S p S

4&5-26

M/M/1 Queue with State-Dependent Rates

Interarrival times: independent, exponential, with parameter λn when at state n

Service times: independent, exponential, with parameter μn when at state n

Service times and interarrival times: independent{N(t): t ≥ 0} is a birth-death processStationary distribution:

0 1 n+1n2

0

1

n

1n

1

2

1n

n

11 1

0 00 1 01 1

, 1 1n n

i in

i n ii i

p p n p

4&5-27 M/M/c Queue

Poisson arrivals with rate λ Exponential service times with parameter μ c servers Arriving customer finds n customers in system

n < c: it is routed to any idle server n ≥ c: it joins the waiting queue – all servers are busy

Birth-death process with state-dependent death rates

[Time spent at state n before jumping to n -1 is the minimum of Bn= min{n,c} exponentials with parameter μ]

, 1

,n

n n c

c n c

0 1 c+1c2

c

2

c

4&5-28 M/M/c Queue

0 1 c+1c2

c

2

c

Detailed balance equations

Normalizing

1 0 0 0

0 0 0

1 ( )1 : ,

( 1) ! !

1 :

! ! !

n n

n n

n c c n c nc c n

n c

cn c p p p p p

n n n n n c

c cn c p p p p p

c c c c c c

1 11 1

00 1 0

( ) ( ) ( ) ( ) 11 1

! ! ! ! 1

k c k cc ck c

nn k k c k

c c c cp p

k c k c

4&5-29 M/M/c Queue

Probability of queueing – arriving customer finds all servers busy

Erlang-C Formula: used in telephony and circuit-switching Call requests arrive with rate λ; holding time of a call

exponential with mean 1/μ c available circuits on a transmission line A call that finds all c circuits busy, continuously attempts to

find a free circuit – “remains in queue” M/M/c/c Queue: c-server loss system

A call that finds all c circuits busy is blocked Erlang-B Formula: popular in telephony

0 0

( ) ( ) 1{queueing}

! ! 1

c cn c

Q nn c n c

c cP P p p p

c c

4&5-30 M/M/c Queue

Expected number of customers waiting in queue – not in service

Average waiting time (in queue)

Average time in system (queued + serviced)

Expected number of customers in system

0 0 2

2

( ) ( )( ) ( )

! ! (1 )

(1 )(1 ) 1

c cn c

Q nn c n c

Q Q

c cN n c p p n c p

c c

P P

(1 )Q

Q

NW P

1 1

(1 )QT W P

(1 )QN T P c

4&5-31 M/M/∞ Queue: Infinite-Server System

Infinite number of servers – no queueing Stationary distribution:

Poisson with rate λ/μ Average number of customers & average delay:

The results hold for an M/G/∞ queue

0 1 n+1n2

( 1)n

2

n

/( / ), 0,1,...

!

n

np e nn

1,

NN T

4&5-32

M/M/c/c Queue: c-Server Loss System

c servers, no waiting room An arriving customer that finds all servers busy is blocked Stationary distribution:

Probability of blocking (using PASTA):

Erlang-B Formula: used in telephony and circuit-switchingResults hold for an M/G/c/c queue

0 1 c2

2

c

1

0

( / ) ( / ), 0,1,...,

! !

n kc

nk

p n cn k

1

0

( / ) ( / )

! !

c kc

ck

pc k

4&5-33

M/M/∞ and M/M/c/c Queues (proof)

DBE:

Normalizing:

0 1 n+1n2

( 1)n

2

n

1 1 2 0

0

( )( 1) ( 1)

( / ), 0,1,...

!

n n n n n

n

n

n p p p p p pn n n n n

p p nn

1

00

( / ), for M/M/c/c

!

kc

k

pk

1

/0

0

( / ), for M/M/c/c

!

k

k

p ek

4&5-34 Sum of IID Exponential RV’s X1, X2,…, Xn: iid, exponential with parameter λ T = X1+ X2+…+ Xn

The probability density function of T is:

[Gamma distribution with parameters (n, λ)]

If Xi is the time between arrivals i -1 and i of a certain type of events, then T is the time until the nth event occurs For arbitrarily small :

Cummulative distribution function:

1( )( ) , 0

( 1)!

nt

T

tf t e t

n

1( ){ arrival occurs in [ , )} ( )

( 1)!

nth t

T

tP n t t f t e

n

1

0

( ){ } 1 { arrival occurs after }

( 1)!

nt s thn

sP t t e ds P n t

n

4&5-35 Sum of IID Exponential RV’s

Example 1: Poisson arrivals with rate λ 1: time until arrival of 1st customer i: ith interarrival time 1, 2,…, n: iid exponential with parameter λ tn= 1+ 2+…,+n: arrival time of customer n

tn follows Gamma with parameters (n, λ).

For arbitrarily small :

1 1

0

( ) ( )( ) , 0; { }

( 1)! ( 1)!

n ntt tn

t tf t e t P t t e dt

n n

1( ){ arrival occurs in [ , )} ( )

( 1)!

nth t

T

tP n t t f t e

n

4&5-36 Sojourn Times in a M/M/1 Queue

M/M/1 Queue – FCFS Ti: time spent in system (queueing + service)

by customer i Ti: exponentially distributed with parameter μ-λ Example of a sojourn time of a customer:

describes the evolution of the queue together with the specific customer

Proof:1. Direct calculation of probability distribution function2. Moment generating functions3. Intuitive: Exercise 3.11(b)

4&5-37 M/M/1 Queue: Sojourn Times (proof)P roof 1: Let ti be the arrival time of customer i, and N i = N (t¡

i ), thenumber of customers in the system right before the ith arrival.

P f Ti > tg =1X

k=0

P f Ti > tjN i = kgP f N i = kg

=1X

k=0

P f D(ti + t) ¡ D(ti ) · kgpk (1)

=1X

k=0

kX

n=0

e¡ ¹ t (¹ t)n

n!¢(1¡ ½)½k (2)

= e¡ ¹ t1X

n=0

(¹ t)n

n!

1X

k=n

(1¡ ½)½k (3)

= e¡ ¹ t1X

n=0

(¹ t)n

n!¢½n = e¡ ¹ t

1X

n=0

(¸t)n

n!(4)

= e¡ ¹ te¸ t = e¡ (¹ ¡ ¸ )t

4&5-38 M/M/1 Queue: Sojourn Times (proof)

P roof 1: Note that:

² Time customer i stays in the system is greater than t, giventhat it ¯nds k customers in the system, i®the number ofdepartures in interval (ti ; ti + t) are less than k + 1. The serveris always busy during that interval, thus times betweendepartures are iid, exponential with parameter ¹ . Then:

P fD(ti + t) ¡ D(ti ) = ng= e¡ ¹ t (¹ t)n

n!; 0 · n · k

² P fNi = kg= pk, by PASTA theorem.

² Eq. (3) follows by changing order of summation.

² Eq. (4) uses:

1X

k=n

½k =1X

k=0

½k ¡n¡ 1X

k=0

½k =1

1¡ ½¡

1¡ ½n

1¡ ½=

½n

1¡ ½

4&5-39 M/M/1 Queue: Sojourn Times (proof)

P roof 2:

² N i : number of customers in system upon arrival of customer i

² T (k)i : sojourn time of customer i when it ¯nds k customers in system

T (k)i = Si + Si ¡ 1 + ::: + Si ¡ k+1 + R i ¡ k

Sj is service time of customer j , and R i ¡ k the residual service timeof the customer in service.

² Si ; : : : ;Si ¡ k+1: iid, exponential with parameter ¹

² R i ¡ k : exponential with parameter ¹ , independent of Si ; : : : ;Si ¡ k+1

² T (k)i is the sum of k iid exponential RV's

² Ti = T (N i )i is the sum of a random number of iid exponential RV's

² Use moment generating functions

4&5-40 M/M/1 Queue: Sojourn Times (proof)

P roof 2:

M T i (t) = E [etT i ] =X 1

k=0E [etT i jN i = k]P f N i = kg

=X 1

k=0E [etT ( k )

i ]pk =X 1

k=0M T ( k )

i(t)pk

A RV is exponential with parameter ¹ if and only if its momentgenerating function is ¹ =(¹ ¡ t).

T (k)i sum of k + 1 independent RV's:

M T ( k )i

(t) = MS i (t)MS i ¡ 1 (t) : : :MS i ¡ k + 1 (t)MR i ¡ k (t) =

µ¹

¹ ¡ t

¶ k+1

Then:

M T i (t) =1X

k=0

µ¹

¹ ¡ t

¶k+1

(1¡ ½)½k = (1¡ ½)¹

¹ ¡ t

1X

k=0

µ¸

¹ ¡ t

¶k

=¹ ¡ ¸¹ ¡ t

11¡ ¸

¹ ¡ t

=¹ ¡ ¸

(¹ ¡ ¸) ¡ t

4&5-41 Moment Generating Function1. De nition: for any t 2 IR:

MX (t) = E [etX ] =

8>><

>>:

Z 1

¡ 1etx f X (x) dx; X continuous

X

j

etx j P fX = xj g; X discrete

2. If the moment generating function MX (t) of X exists and is ¯nite insome neighborhood of t = 0, it determines the distribution of Xuniquely.

3. Fundamental Properties: for any n 2 IN :

(i)dn

dtn MX (t) = E [X n etX ]

(ii)dn

dtn MX (0) = E [X n ]

4. Moment Generating Functions and Independence:

X ;Y : independent ) MX +Y (t) = MX (t)M Y (t)

The opposite is not true.