1

Birth and death process

N(t) Depends on how fast arrivals or departures occur

Objective

N(t) = # of customersat time t.

λ

arrivals(births)

departures(deaths)

μ

))(Pr( itN

2

Behavior of the system

λ>μ

λ<μ

Possible evolution of N(t)

Time 1 2 3 4 5 6 7 8 9 10 11

123

busy idleN(t)

3

General arrival and departure rates

λn

Depends on the number of customers (n) in the system

Example

μn

Depends on the number of customers in the system

Example

Mn

Mnn ,0

,

n

Or

n

n

n

4

Changing the scale of a unit time

Number of arrivals/unit time Follows the Poisson distribution with rate λn

Inter-arrival time of successive arrivals is exponentially distributed

Average inter-arrival time = 1/ λn

What is the avg. # of customers arriving in dt?

Time

min/2/1min/16.030/30:

hourExample

dtAverage

n

n

5

Probability of one arrival in dt

dt so small Number of arrivals in dt, X is a r.v.

X=1 with probability p

X=0 with probability 1-p

Average number of arrivals in dt

Prob (having one arrival in dt) = λn dt

dtXEbut

pppXE

n][,

)1.(0.1][

dt

6

Probability of having 2 events in dt

Departure rate in dt μn dt

Arrival rate in dt λn dt

What is the probability Of having an (arrival+departure), (2 arrivals or departures)

222

222

2

)()()2Pr(

)()()2Pr(

)(.)11Pr(

dtdtdepartures

dtdtarrivals

dtdtdtdeparturearrival

nn

nn

nnnn

7

Probability distribution of N(t)

Pn (t) The probability of getting n customers by time t

The distribution of the # of customers in systemt+dtt

? nn-1: arrivaln+1: departuren: none of the above

)1)((

)()()( 1111

dtdttP

dttPdttPdttP

nnn

nnnnn

8

Differential equation monitoring evolution of # customers

?)(

))(()()()(

))(()()()()(

)1)(()()()(

1111'

1111

1111

tP

tPtPtPtP

tPtPtPdt

tPdttP

dtdttPdttPdttPdttP

n

nnnnnnnn

nnnnnnnnn

nnnnnnnn

These are solved Numerically using MATLAB

We will explore the cases Of pure death

And pure birth

9

Pure birth process

In this case μn =0, n >= 0

λn = λ, n >= 0

!

)()(;)(

0,)()()(

0),(.)(

0,)()()(

0

1'

0'0

1'

n

ettPetP

ntPtPtP

ntPtP

ntPtPtP

tn

nt

nnn

nnn

Hence,

First order differential equation

10

11

Pure death process

In this case λn =0, n >= 0

μn = μ

)!(

)()(;)(

),(.)(

0,)()()(

0,)()()(

'

1'

1'

nM

ettPetP

MntPtP

ntPtPtP

ntPtPtP

tnM

nt

M

MM

nnn

nnn

12

Queuing system

Transient phase

Steady state Behavior is independent of t

Pn (t)

λ μ

nnt

PtP

)(lim

Pn (t)

t

transient Steady state

13

Differential equation: steady state analysis

Limiting case

01

01

1111

1100

1111

'

0,)(

0,)(0

0)(

)(

lim

lim

PP

nPPP

PP

nPPP

tP

PtP

nnnnnnn

nnnnnnn

nt

nnt

14

Solving the equations

n=1

n=22200111 )( PPP

3311222 )( PPP

(1)

(1) =>

021

102

12

122211

22111111

PP

PPPP

PPPP

15

Pn

What about P0

1,...

...0

21

110

0321

2102

23

233322

33222222

nPP

PP

PPPP

PPPP

n

nn

16

Normalization equation

1

1

1

0

0

021

100

1

00

10

1

1

1....

1.......

ni

n

i

i

n

i

n

P

PPP

ionNormilizat

PPP

17

Conditional probability and conditional expectation: d.r.v.

X and Y are discrete r.v. Conditional probability mass function

Of X given that Y=y

Conditional expectation of X given that Y=y

)(

),(

)(

),(

)|()|(|

yp

yxp

yYP

yYxXP

yYxXPyxp

Y

YX

x

yYxXPxyYXE )|(.]|[

18

Conditional probability and expectation: continuous r.v.

If X and Y have a joint pdf fX,Y(x,y) Then, the conditional probability density function

Of X given that Y=y

The conditional expectation Of X given that Y=y

)(

),()|(| yf

yxfyxf

YYX

dxyxfxyYXE YX )|(.]|[ |

19

Computing expectations by conditioning Denote

E[X|Y]: function of the r.v. Y Whose value at Y=y is E[X|Y=y]

E[X|Y]: is itself a random variable Property of conditional expectation

if Y is a discrete r.v.

if Y is continuous with density fY (y) =>

]]|[[][ YXEEXE

y

yYPyYXEYXEEXE ][].|[]]|[[][

dyyfyYXEXE Y )(]|[][

(1)

(2)

(3)

20

Proof of equation when X and Y are discrete

][][

],[],[

][][

],[

][].|[

][].|[]]|[[

XExXxP

yYxXPxyYxXxP

yYPyYP

yYxXPx

yYPyYxXxP

yYPyYXEYXEE

x

x yy x

y x

y x

y

21

Problem 1 Sam will read

Either one chapter of his probability book or

One chapter of his history book

If the number of misprints in a chapter Of his probability book

is Poisson distributed with mean 2

Of his history book is Poisson distributed with mean 5

Assuming Sam equally likely to choose either book What is the expected number of misprints he comes across?

22

Solution

2

7)2

1(2)

2

1(5

]2[].2|[]1[].1|[][

_,2

__,1

_

YPYXEYPYXEXE

chosenyprobabilit

chosenbookhistoryY

mistakesnumberX

23

Problem 2 A miner is trapped in a mine containing three doors

First door leads to a tunnel that takes him to safety

After 2 hours of travel

Second door leads to a tunnel that returns him to the mine

After 3 hours of travel

Third door Leads to a tunnel that returns him to the mine

After 5 hours Assuming he is equally likely to choose any door

What is the expected length of time until he reaches safety?

24

Solution

10][][5][32(3

1][

][5]3|[

];[3]2|[;2)1|[

])3|[]2|[]1|[(3

1

]3[]3|[

]2[]2|[]1[]1|[][

__

__

XEXEXEXE

XEYXE

XEYXEYXE

YXEYXEYXE

YPYXE

YPYXEYPYXEXE

choseninitiallydoorY

safetyuntiltimeX

25

Computing probabilities by conditioning Let E denote an arbitrary event

X is a random variable defined by

It follows from the definition of X

tdoesnEif

occursEifX

'__,0

__,1

continuousYdyyfyYEPEP

discreteYyYPyYEPEP

yYEPyYXEEPXE

Y

y

_,)()|()(

_,)().|()(

)3(&)2(

)|(]|[)(][

26

Problem 3

Suppose that the number of people Who visit a yoga studio each day

is a Poisson random variable with mean λ

Suppose further that each person who visit is, independently, female with probability p

Or male with probability 1-p

Find the joint probability That n women and m men visit the academy today

27

Solution

Let N1 denote the number of women, N2 the number of men

Who visit the academy today

N= N1 +N2 : total number of people who visit Conditioning on N gives

Because P(N1=n,N2=m|N=i)=0 when i != n+m

0

2121 )().|,(),( iNPiNmNnNPmNnNP

)!().|,(

)().|,(),(

21

2121

mnemnNmNnNP

mnNPmnNmNnNPmNnNPmn

28

Solution (cont’d)

Each of the n+m visit is independently a woman with probability p

The conditional probability That n of them are women is

The binomial probability of n successes in n+m trials

!

))1((

!

)(

)!()1(),(

)1(

21

m

pe

n

pe

mnepp

n

mnmNnNP

mp

np

mnmn

29

Solution: analysis

When each of a Poisson number of events is independently classified

As either being type 1 with probability p

Or type 2 with probability (1-p)

=> the numbers of type 1 and 2 events Are independent Poisson random variables

!

))1(()(

!

)(),()(

)1(2

0211

m

pemNP

and

n

pemNnNPnNP

mp

m

np

30

Problem 4

At a party N men take off their hats

The hats are then mixed up and Each man randomly selects one

A match occurs if a man selects his own hat

What is the probability of no matches?

31

Solution E = event that no matches occur

P(E) = Pn : explicit dependence on n

Start by conditioning Whether or not the first man selects his own hat

M: if he did, Mc : if he didn’t

P(E|Mc) Probability no matches when n-1 men select of n-1

That does not contain the hat of one of these men

n

nMEPPMEP

MPMEPMPMEPEPP

cn

ccn

1)|(0)|(

);()|()()|()(

32

Solution (cont’d)

P(E|Mc) Either there are no matches and

Extra man does not select the extra hat

=> Pn-1 (as if the extra hat belongs to this man)

Or there are no matches Extra man does select the extra hat

=> (1/n-1)xPn-2

21

21

111

1)|(

nnn

nnc

Pn

Pn

nP

Pn

PMEP

33

Solution (cont’d)

Pn is the probability of no matches When n men select among their own hats

=> P1 =0 and P2 = ½

=>

!

)1(....

!4

1

!3

1

!2

1

!4

1

!3

1

!2

1;!3

1

!2

143

nP

PP

n

n

34

Problem 5: continuous random variables

The probability density function of a non-negative random variable X is given by

Compute the constant λ?

10.)(x

X exf

10

110).10(1

)(1

0

10/

0

10/

x

xX

e

dxedxxf

35

Problem 6: continuous random variables Buses arrives at a specified stop at 15 min intervals

Starting at 7:00 AM They arrive at 7:00, 7:15, 7:30, 7:45

If the passenger arrives at the stop at a time Uniformly distributed between 7:00 and 7:30

Find the probability that he waits less than 5 min? Solution

Let X denote the number of minutes past 7 That the passenger arrives at the stop =>X is uniformly distributed over (0, 30)

3

1

30

1

30

1

)3025Pr()1510Pr(30

25

15

10

dxdx

XX

36

Problem 7: conditional probability

Suppose that p(x,y) the joint probability mass function of X and Y is given by P(0,0) = .4, P(0,1) = .2, P(1,0) = .1, P(1,1) = .3

Calculate the conditional probability mass function of X given Y = 1

)(

),(

)(

),(

)|()|(|

yp

yxp

yYP

yYxXP

yYxXPyxp

Y

YX

5

3

)1(

)1,1()1|1(,

5

2

)1(

)1,0()1|0(,

5.0)1,1()1,0()1,()1(

|

|

YYX

YYX

xY

P

PPand

P

PPhence

PPxPP

37

counting process A stochastic process {N(t), t>=0}

is said to be a counting process if N(t) represents the total number of events that occur by time t

N(t) must satisfy N(t) >= 0

N(t) is integer valued

If s < t, then N(s) <= N(t)

For s < t, N(s) – N(t) = # events in the interval (s,t]

Independent increments # of events in disjoint time intervals are independent

38

Poisson process

The counting process {N(t), t>=0} is Said to be a Poisson process having rate λ, if

N(0) = 0

The process has independent increments

The # of events in any interval of length t is Poisson distributed with mean λt, that is

,...1,0,!

)())()(( n

n

tensNstNP

nt

39

Properties of the Poisson process

Superposition property If k independent Poisson processes

A1, A2, …, An

Are combined into a single process A

=> A is still Poisson with rate Equal to the sum of individual λi of Ai

40

Properties of the Poisson process (cont’d)

Decomposition property Just the reverse process

“A” is a Poisson process split into n processes Using probability Pi

The other processes are Poisson With rate Pi.λ