Queueing Theory I

Summary

Little’s Law Queueing System Notation Stationary Analysis of Elementary Queueing

Systems M/M/1 M/M/m M/M/1/K …

Little’s Law

a(t): the process that counts the number of arrivals up to t. d(t): the process that counts the number of departures up to t. N(t)= a(t)- d(t)

N(t)

a(t)

Time t

Area γ(t)

Average arrival rate (up to t) λt= a(t)/t

Average time each customer spends in the system Tt= γ(t)/a(t)

Average number in the system Nt= γ(t)/t

d(t)

Little’s Law

t t tN T Taking the limit as t goes to infinity

E EN TExpected number of customers in the system

Expected time in the system

Arrival rate IN the system

N(t)

a(t)

Time t

Area γ(t)

d(t)

Generality of Little’s Law

Little’s Law is a pretty general result It does not depend on the arrival process distribution It does not depend on the service process distribution It does not depend on the number of servers and buffers

in the system.

E EN T

Queueing Network

Queueing Network

λ

Aggregate Arrival rate

Specification of Queueing Systems

Customer arrival and service stochastic models Structural Parameters

Number of servers Storage capacity

Operating policies Customer class differentiation (are all customers

treated the same or do some have priority over others?) Scheduling/Queueing policies (which customer is

served next) Admission policies (which/when customers are

admitted)

Queueing System Notation

A/B/m/K/N

Arrival Process•M: Markovian •D: Deterministic•Er: Erlang•G: General

Service Process•M: Markovian •D: Deterministic•Er: Erlang•G: General

Number of servers m=1,2,…

Storage Capacity K= 1,2,… (if ∞ then it is omitted)

Number of customers N= 1,2,… (for closed networks otherwise it is omitted)

Performance Measures of Interest

We are interested in steady state behavior Even though it is possible to pursue transient results, it is a

significantly more difficult task. E[S] average system time (average time spent in the

system) E[W] average waiting time (average time spent waiting

in queue(s)) E[X] average queue length E[U] average utilization (fraction of time that the

resources are being used) E[R] average throughput (rate that customers leave the

system) E[L] average customer loss (rate that customers are lost

or probability that a customer is lost)

Recall the Birth-Death Chain Example

At steady state, we obtain

λ0

0 1μ1

λ1

2μ2

λj-2

j-1μj-1

λj-1

jμjμ3

λ2λj

μj+1

0 0 1 1 0 01 0

1

In general

1 1 1 1 0j jj j j j j 0

1 01 1

...

...j

jj

Making the sum equal to 1

0 1

01 1

...1 1

...j

j j

Solution exists if

0 1

1 1

...1

...j

j j

S

M/M/1 Example

Meaning: Poisson Arrivals, exponentially distributed service times, one server and infinite capacity buffer.

λ

0 1μ

λ

2μ

λ

j-1μ

λ

jμμ

λ λ

μ

Using the birth-death result λj=λ and μj=μ, we obtain

0 , 0,1,2,...j

j j

Therefore

01

11j

j

for λ/μ = ρ <1

0 1

, 1, 2,...1 jj j

M/M/1 Performance Metrics

Server Utilization

Throughput

01

1 1 1jj

E U

Expected Queue Length

01

1jj

E R

0 0 0

1 1j

jj

j j j

dE j jX

d

0

11 1

1 1j

j

d d

d d

M/M/1 Performance Metrics

Average System Time

Average waiting time in queue

1E E E ES SX X

E E E E E ES W W SZ Z

1 1

1 1E S

1 1

1 1E W

M/M/1 Performance Metrics Examples

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

5

10

15

20

25

30

35

40

rho

Del

ay (

time

units

) /

Num

ber

of c

usto

mer

s

μ=0.5

Ε[Χ]

Ε[W]

Ε[S]

PASTA Property

PASTA: Poisson Arrivals See Time Averages

Let πj(t)= Pr{ System state X(t)= j }

Let aj(t)= Pr{ Arriving customer at t finds X(t)= j }

In general πj(t) ≠ aj(t)! Suppose a D/D/1 system with interarrival times equal to 1 and

service times equal to 0.5

0 0.5 1.0 1.5 2.0 2.5 3.0

a a a a

Thus π0(t)= 0.5 and π1(t)= 0.5 while a0(t)= 1 and a1(t)= 0!

Theorem

For a queueing system, when the arrival process is Poisson and independent of the service process then, the probability that an arriving customer finds j customers in the system is equal to the probability that the system is at state j. In other words,

Pr , 0,1,...j ja X j jt t t Proof:

0

lim Pr | ,j ta X j a t t tt t

Arrival occurs in interval (t,Δt)

0

Pr , ,lim

Pr ,t

X j a t t tt

a t t t

0

Pr Pr ,lim Pr

Pr ,jt

X j a t t ttX jt t

a t t t

M/M/m Queueing System

Meaning: Poisson Arrivals, exponentially distributed service times, m identical servers and infinite capacity buffer.

λ

0 1μ

λ

22μ

λ

mmμ

λ

m+1mμ3μ

λ λ

mμ

1

m…

if 0 and =

if j j

j j m

m j m

M/M/m Queueing System

Using the general birth-death result

0

1, if

!

j

j j mj

Letting ρ=λ/(mμ) we get

0 , if !

jm

j

mj m

mm

0

0

if !

if !

j

jm j

mj m

j

mj m

m

1

01

11! !

j m jm

j j m

m m

j m

11

01

1! ! 1

j mm

j

m m

j m

To find π0

M/M/m Performance Metrics

Server Utilization

1

1

Prm

jj

E j m X mU

1

01 ! !

j m jm

j j m

m mj m

j m

1

02 ! !1 1

j mm

j

m mmm

mj

1 1 1 11

02

1! ! ! !1 11 1

j m m mm

j

mm m m mm

mj m m

1

01

1! ! 1

j mm

j

m mm

j m

00

1m m

M/M/m Performance Metrics

Throughput

Expected Queue Length

1

1

m

j jj j m

E j mR

1

00 1

...! !

j mmj

jj j j m

m mE j j jX

j m

02! 1

mm

E mXm

Using Little’s Law

02

1 1

! 1

mm

E E mS Xm

Average Waiting time in queue 1E EW S

M/M/m Performance Metrics

Queueing Probability

00Pr

! ! 1

mm j

Q jj m j m

mmP X m

m m

Erlang C Formula

Example

Suppose that customers arrive according to a Poisson process with rate λ=1. You are given the following two options, Install a single server with processing capacity μ1= 1.5

Install two identical servers with processing capacity μ2= 0.75 and μ3= 0.75

Split the incoming traffic to two queues each with probability 0.5 and have μ2= 0.75 and μ3= 0.75 serve each queue.

μ1λ

Αμ2

μ3

λ

Β

μ2

μ3

λ

C

Example

Throughput It is easy to see that all three systems have the same throughput

E[RA]= E[RB]= E[RC]=λ

Server Utilization

1

1 2

1.5 3AE U

2

1 4

0.75 3BE U

Therefore, each server is 2/3 utilized

2

0.5 1 2

2 0.75 3CE U

Therefore, all servers are similarly loaded.

Example

Probability of being idle

01

11

3A

For each server02

11

2 3C

124

14 31523 2 1

3

11

01

1! ! 1

j mm

j

m m

j m

Example

Queue length and delay

1

12

1.5 1AE X

For each queue!

02

12

! 51

m

B

mE mX

m

12

/ 2 0.52

/ 2 0.75 0.5CE X

12A AE ES X

12 4C CE X E X

1 12

5B BE ES X

14C CE X E X

M/M/∞ Queueing System

Special case of the M/M/m system with m going to ∞λ λ λ λ

0 1μ 22μ

λ

mmμ m+1(m+1)μ3μ

λ

and = for all j j j j

0!

j

j j

Let ρ=λ/μ then, the state probabilities are given by

0 01

1 1!

j

j

ej

!

j

j

e

j

System Utilization and Throughput

01 1E eU E R

M/M/∞ Performance Metrics

Expected Number in the System

1

0 0 1! !1

j j

jj j j

E j j e eXj j

Using Little’s Law

1 1 1E ES X

No queueing!

Number of busy servers

M/M/1/K – Finite Buffer Capacity

Meaning: Poisson Arrivals, exponentially distributed service times, one server and finite capacity buffer K.

Using the birth-death result λj=λ and μj=μ, we obtain

0 , 0,1,2,...j

j j K

Therefore

01

11jK

j

for λ/μ = ρ

0 1

1

1 K

1

1, 1, 2,...

1

j

j Kj K

λ

0 1μ

λ

2μ

λ

K-1μ

λ

Kμμ

λ

M/M/1/K Performance Metrics

Server Utilization

Throughput

0 1 1

1 11 1

1 1

K

K KE U

Blocking Probability

0 1

11

1

K

KE R

1

1

1

K

B K KP

Probability that an arriving customer finds the queue full (at state K)

M/M/1/K Performance Metrics

Expected Queue Length

1 10 0 0

1 1

1 1

jK K Kj

j K Kj j j

dE j jX

d

1

1 10

1 1 1

1 1 1

KKj

K Kj

d d

d d

1

21

111 1

1 1

KK

K

K

1

1

1 1

KK

KK

System time

1 KE E SX

Net arrival rate (no losses)

M/M/m/m – Queueing System

Meaning: Poisson Arrivals, exponentially distributed service times, m servers and no storage capacity.

Using the birth-death result λj=λ and μj=μ, we obtain

0

1, 0,1,2,...

!

j

j j mj

Therefore

00

11

!

jm

j j

for λ/μ = ρ

1

00 !

jm

j j

0 , 1, 2,...!

j

j j mj

λ

0 1μ

λ

22μ

λ

m-1(m-1)μ

λ

mmμ3μ

λ

M/M/m/m Performance Metrics

Throughput

Blocking Probability

!0

/ !11 j

m

mmjj

mE R

!0

/ !j

m

B m m

jj

mP

Probability that an arriving customer finds all servers busy (at state m)

Erlang B Formula

M/M/1//N – Closed Queueing System

Meaning: Poisson Arrivals, exponentially distributed service times, one server and the number of customers are fixed to N.

Nλ

0 1μ

(N-1)λ

2μ

2λ

N-1μ

λ

Nμμ

(N-2)λ

λ

μ

1

N

…

Using the birth-death result, we obtain

0

!, 1, 2,...

!j

j

Nj N

N j

1

00

!

!

Nj

j

N

N j

M/M/1//N – Closed Queueing System

Response Time Time from the moment the customer entered the queue until it

received service.

01E E SX

Nλ

0 1μ

(N-1)λ

2μ

2λ

N-1μ

λ

Nμμ

(N-2)λ

For the queue, using Little’s law we get,

0

11E N X

In the “thinking” part,

0

0 0

11 1

1 1

N NE S

Therefore