Math/Stat 416 Homework 9

1. Let Xn be a discrete time Markov chain with probability transitionmatrix

P =

0.5 0.3 0.20.2 0.6 0.20.2 0.4 0.4

and N(t) is a Poisson process with λ = 3. XN(t) is now a continuoustime Markov chain.

(a) Find the P ∗i,j and vi for XN(t).

(b) Find the transition rate matrix Q for XN(t)

(c) Find the limiting distribution for Xn and for XN(t). Commenton their similarity or lack thereof.

(d) Suppose that there is a different Poisson process M(t) with λ = 1.Construct the probability transition matrix of a discrete Markovchain Yn such that YM(t) has the same transition rate matrix(called Q above) as XN (t).

2. A population of organisms consists of both male and female mem-bers. In a small colony any particular male is likely to mate with anyparticular female in any time interval of length h, with probabilityλh + o(h). Each mating immediately produces one offspring, equallylikely to be male or female. Let X1(t) and X2(t) denote the number ofmales and females in the population at t. Derive the transition ratesof the continuous time Markov chain (X1(t), X2(t)) and also derive theappropriate P ∗ and vi values.

3. There are N individuals in a population, some of whom have a certaininfection that spreads as follows. Contacts between two members ofthis population occur in accordance with a Poisson process havingrate λ. When a contact occurs, it is equally likely to involve any ofthe

(N2

)pairs of individuals in the population. If a contact involves

1

an infected and a noninfected individual, then with probability p thenoninfected individual becomes infected. Once infected, an individualremains infected from that time on. Let X(t) denote the number ofinfected members of the population at time t. Specify the transitionrates and the P ∗ and vi values.

4. Each individual in a biological population is assumed to give birthat an exponential rate λ and to die and an exponential rate µ. Inaddition there is an exponential rate θ due to immigration. However,immigration is not allowed when the population size is N or larger.

(a) Set this up as a birth/death process.

(b) If N = 3, θ = λ = 1, µ = 2, determine the proportion of time thatimmigration is restricted.

5. Potential customers arrive at a full-service, one-pump gas station at aPoisson rate of 20 cars per hour. However, customers will only enterthe station for gas if there are no more than two cars (including theone currently being attended to) at the pump. Suppose the amount oftime required to service a car is exponentially distributed with a meanof five minutes.

(a) What fraction of the attendant’s time will be spent servicing cars?

(b) What fraction of potential customers are lost?

6. Customers arrive at a single-server queue in accordance with a Poissonprocess having rate λ. However, an arrival that finds n customersalready in the system will only join the system with probability 1/(n+1). (Otherwise, i.e. with probability n/(n+1), that arriving customersimply doesn’t join the queue.) Find the limiting distribution of thenumber of customers in the system.

2