MTH 412a: Applied Stochastic ProcessesAssignment No. 5: Continuous Time Markov Chains

1. Let {X(t) : t ≥ 0} be a pure birth process with λn = λ, n = 0, 1, 2, . . . andP (X(0) = 0) = 1, where λ > 0. If p(t) = P (X(t) = j), t ≥ 0, j = 0, 1, 2, . . .,then show that p(t) = e−λt(λt)j/j!.

2. Consider a Yule process with X(0) = 1. Given X(t) = n+1, show that the birthtimes S1, . . . , Sn are distributed as order statistics based on a random sampleof size n from a distribution having pdf f(x) = ce−λ(t−x), 0 ≤ x < t. Find thenormalizing constant c.

3. A machine, whose lifetime is exponentially distributed with rate λ > 0 is putinto service. Each time machine breaks down, it is immediately put into repair.Assume that the repair time is exponentially distributed with rate µ > 0. Giventhat the machine is up at time t = 0, find the probability that it will be up attime t = 10.

4. For a pure birth process compute the mean, variance and m.g.f. of the time ittakes the population to go from size 0 to size N .

5. Consider the Yule process with X(0) = i. Given that X(t) = i+k, what can besaid about the conditional distribution of the birth times of the k individualsin (0, t)?

6. For a continuous time Markov chain {X(t) : t ≥ 0}, let p(t) = p0,0(t).

(a) Find limt→01−p(t)

t;

(b) Show that p(t)p(s) ≤ p(t+ s) ≤ 1− p(s) + p(s)p(t), s ≥ 0, t ≥ 0;(c) Show that |p(t)− p(s)| ≤ 1− p(t− s), ∀0 < s < t and conclude that p(·) iscontinuous.

7. For the Yule process suppose that X(0) = 1 and that at time T the processstops and is replaced by an emigration process in which departure occur inaccordance with a Poisson process of rate µ. Let τ be the time taken after Tfor the population to vanish. Find the pdf of τ and show that E(τ) = eλT/µ.

8. Consider a birth and death process with rates {λn}n≥0 and {µn}n≥1. Startingin state i, find the probability that the first k events are all births.

9. Consider two machines, each having exponential lifetime with rate λ. There isa single repairman that can service machine at an exponential rate µ. Set upthe Kolmogorov backward equations.

10. For the pure death process with µn = µ, n ≥ 1, find pij(t), i, j ≥ 0.

11. Consider a birth and death process with λi = (i+1)λ, i ≥ 0, and µi = iµ, i ≥ 1.(a) Determine the expected time to go from state 0 to state 4;(b) Determine the expected time to go from state 2 to state 5;(c) Determine the variances in parts (a) and (b).

12. Each individual in a biological population is assumed to give births at an ex-ponential rate λ and to die at an exponential rate µ. In addition, there is anexponential rate of increase due to immigration. However, immigration is notallowed when the population size is N or large. Set this as a birth and deathmodel.

13. Let {X(t) : t ≥ 0} be a continuous time stationary Markov chain with statespace S = {−1, 1}, initial distribution p1 = P (X(0) = 1) = p(X(0) = −1) =p−1 = 0.5, p1,1(t) = p−1,−1(t) = α(t), t ≥ 0, where α(·) is a continuous functionon [0,∞) with α(0) = 1.(a) Find P (X(t) = 1|X(0) = 1), t ≥ 0;(b) Show that α(t) = (1 + e−λt)/2, t ≥ 0, for some λ > 0.

14. A small barbershop, operated by a single barber, has room for at most twocustomers. Potential customers arrive at a Poisson rate of three per hour, andsuccessive service times are independent and identically distributed randomvariables with rate 4. What is(a) the average number of customers in the shop?;(b) the proportion of customers that enter the shop?;(c) If the barber could work twice at fast, how much more business he woulddo?

15. Potential customers arrive at a full-service, one-pump gas station at a Poissonrate of 20 cars per hour. However, customers will only enter the station for gasif there are no more than two cars (including the one currently being attendedto) at the pump. Suppose the amount of time required to service a car isexponentially distributed with a mean of five minutes.(a) What fraction of the attendant’s time will be spent servicing cars?(b) What fraction of potential customers are lost?

16. A service center consists of two servers, each working at an exponential rateof two service per hour. If customers arrive at Poisson rate of three per hour,then, assuming a system capacity of at most three customers,(a) what fraction of potential customers enter the system?(b) what would the value of part (a) be if there was only a single server, andhis rate was twice as fast (that is, µ = 4)?

17. A single repair person looks after both machines 1 and 2. Each time it isrepaired, machine i stays up for an exponential time with rate λi, i = 1, 2.When machine i fails, it requires an exponentially distributed amount of workwith rate µi to complete the repair. The repair person will always servicemachine 1 when it is down. For instance, if machine 1 fails while 2 is beingrepaired, then the repair person will immediately stop work on machine 2 andstart on 1. What proportion of time machine 2 is down?