MTHE/STAT455, STAT855 Fall 2017

Applied Stochastic Processes

Final Exam Practice Problems

1. The M/M/s queue is a queueing model in which customers arrive according to aPoisson process with rate λ. There are s servers and all service times are independentand identically distributed Exponential(µ) random variables. An arriving customerenters service immediately if there is at least one server free, and otherwise goes intoa queue. The service discipline is first come first served. When a customer completesservice that customer leaves the system. Let X(t) be the number of customers in thesystem (in service or in queue) at time t, for t ≥ 0. Then {X(t) : t ≥ 0} is a continuoustime Markov chain. Write down the entries of the infinitesimal generator matrix andfind the stationary distribution.

2. Consider a simple random walk modified to be a continuous time process as follows.Jumps occur at the event times of a Poisson process with rate λ, and at each jumptime the walk jumps up with probability p and down with probability 1− p. Assumeit starts in state 0 at time 0.

(a) For given integers k1 ≥ 0 and k2 ≥ 0, what is the probability that this walk makesexactly k1 up moves and k2 down moves by time t?

(b) What are E[X(t)] and Var(X(t)), where X(t) is the position of the walk at timet? Hint: write

X(t) =

N(t)∑i=1

Yi,

where {N(t) : t ≥ 0} is the Poisson process and Yi is the ith jump (Yi = 1 withprobability p and Yi = −1 with probability 1− p).

(c) What is the probability that the walk makes 0 jumps by time t, 3 jumps by time2t and is in state 1 at time 2t?

3. (a) Let X1, X2, . . . , be independent and identically distributed Geometric(p) randomvariables, and suppose the Xi’s are successively observed (i.e., Xn is observed attime n). Let Nk be the first time that k consecutive observations are greater thanm, where m is a fixed positive integer.

(i) What is P (Nk = k)?

(ii) What are E[N1] and E[N2]? (Hint: Relate to a coin tossing experiment).

(iii) For k ≥ 3, express E[Nk] in terms of E[Nk−1] and determine E[Nk] (in termsof p and k).

MTHE/STAT455, STAT855 -- Midterm Exam Practice Problems, 2017 Page 2 of 3

(b) Suppose that n dice are rolled. One of the dice (say the first one) is fair, while theothers are biased. Let N denote the number of dice that show the same numberas the fair die (including the fair die). Find E[N ].

4. (a) Let {Xn : n ≥ 0} be a discrete time Markov chain with state space S ={0, 1, 2, . . .} and transition probabilities pi,i+1 = p and pi,0 = q = 1 − p for alli ∈ S, where 0 < p < 1. All other transition probabilities are equal to zero. Givethe classes of this Markov chain, the period of each class, and whether each classis recurrent or transient. If a stationary distribution exists, find it.

(b) Suppose that {Yn : n ≥ 0} is an irreducible, aperiodic, positive recurrent, discretetime Markov chain with state space S and suppose there exists a state i ∈ S suchthat pji = q for all j ∈ S, where 0 < q < 1 and pji is the transition probability fromstate j to state i. Show that, starting in state i, the mean number of transitionsto return to state i is 1/q.

5. Consider a service system with infinite capacity. Customers arrive to the system ac-cording to a Poisson process with rate λ. Each arriving customer immediately beginsprocessing, is processed in the system for exactly 30 time units and then leaves thesystem. The unfinished work in the system at time t, U(t), is defined to be the totalamount of processing left to be done for the customers currently in the system at timet. Let N(t) denote the total number of customers that have arrived to the system bytime t.

(a) Show that

E[U(t)∣∣ N(t) = n] = E

[n∑

i=1

(30− (t− Si))I[t−30,t](Si)∣∣∣ N(t) = n

],

where S1, S2, . . . , Sn are the arrival times of the first n customers.

(b) Show that E[U(t)] = 450λ when t ≥ 30.

(c) Show that E[U(t)] = 30λt− 12λt2 when t < 30.