# Chapter 3: Markov Processes First hitting times · PDF fileChapter 3: Markov Processes First...

date post

18-Jun-2018Category

## Documents

view

222download

0

Embed Size (px)

### Transcript of Chapter 3: Markov Processes First hitting times · PDF fileChapter 3: Markov Processes First...

Chapter 3: Markov ProcessesFirst hitting times

L. BreuerUniversity of Kent, UK

November 3, 2010

L. Breuer Chapter 3: Markov Processes First hitting times

Example: M/M/c/c queue

Arrivals: Poisson process with rate > 0

Service times: exponentially distributed with rate > 0c serversno waiting roomIf upon an arrival the system is filled, i.e. all servers are busy, thisarriving user is not admitted into the system. In this case we saythat the arriving user is lost. Queueing systems with the possibilityof such an event are thus called loss systems.

L. Breuer Chapter 3: Markov Processes First hitting times

Example: M/M/c/c queue

Arrivals: Poisson process with rate > 0Service times: exponentially distributed with rate > 0

c serversno waiting roomIf upon an arrival the system is filled, i.e. all servers are busy, thisarriving user is not admitted into the system. In this case we saythat the arriving user is lost. Queueing systems with the possibilityof such an event are thus called loss systems.

L. Breuer Chapter 3: Markov Processes First hitting times

Example: M/M/c/c queue

Arrivals: Poisson process with rate > 0Service times: exponentially distributed with rate > 0c servers

no waiting roomIf upon an arrival the system is filled, i.e. all servers are busy, thisarriving user is not admitted into the system. In this case we saythat the arriving user is lost. Queueing systems with the possibilityof such an event are thus called loss systems.

L. Breuer Chapter 3: Markov Processes First hitting times

Example: M/M/c/c queue

Arrivals: Poisson process with rate > 0Service times: exponentially distributed with rate > 0c serversno waiting room

If upon an arrival the system is filled, i.e. all servers are busy, thisarriving user is not admitted into the system. In this case we saythat the arriving user is lost. Queueing systems with the possibilityof such an event are thus called loss systems.

L. Breuer Chapter 3: Markov Processes First hitting times

Example: M/M/c/c queue

Arrivals: Poisson process with rate > 0Service times: exponentially distributed with rate > 0c serversno waiting roomIf upon an arrival the system is filled, i.e. all servers are busy, thisarriving user is not admitted into the system.

In this case we saythat the arriving user is lost. Queueing systems with the possibilityof such an event are thus called loss systems.

L. Breuer Chapter 3: Markov Processes First hitting times

Example: M/M/c/c queue

Arrivals: Poisson process with rate > 0Service times: exponentially distributed with rate > 0c serversno waiting roomIf upon an arrival the system is filled, i.e. all servers are busy, thisarriving user is not admitted into the system. In this case we saythat the arriving user is lost.

Queueing systems with the possibilityof such an event are thus called loss systems.

L. Breuer Chapter 3: Markov Processes First hitting times

Example: M/M/c/c queue

Arrivals: Poisson process with rate > 0Service times: exponentially distributed with rate > 0c serversno waiting roomIf upon an arrival the system is filled, i.e. all servers are busy, thisarriving user is not admitted into the system. In this case we saythat the arriving user is lost. Queueing systems with the possibilityof such an event are thus called loss systems.

L. Breuer Chapter 3: Markov Processes First hitting times

Example: M/M/c/c queue (contd.)

The queue described above is a skip-free Markov process Y withstate space E = {0, . . . , c}

and generator matrix

Q =

2 2 . . .

. . .. . .

(c 1) (c 1) c c

up to the first loss.

L. Breuer Chapter 3: Markov Processes First hitting times

Example: M/M/c/c queue (contd.)

The queue described above is a skip-free Markov process Y withstate space E = {0, . . . , c} and generator matrix

Q =

2 2 . . .

. . .. . .

(c 1) (c 1) c c

up to the first loss.

L. Breuer Chapter 3: Markov Processes First hitting times

Example: M/M/c/c queue (contd.)

We wish to know the time until the first loss.

Consider the Markovprocess Y with state space E = {0, . . . , c + 1} and generatormatrix

Q =

2 2 . . .

. . .. . .

c c 0 . . . . . . 0 0

Now

Z := inf{t 0 : Y t = c + 1}

is the random variable of the time until the first loss.

L. Breuer Chapter 3: Markov Processes First hitting times

Example: M/M/c/c queue (contd.)

We wish to know the time until the first loss. Consider the Markovprocess Y with state space E = {0, . . . , c + 1}

and generatormatrix

Q =

2 2 . . .

. . .. . .

c c 0 . . . . . . 0 0

Now

Z := inf{t 0 : Y t = c + 1}

is the random variable of the time until the first loss.

L. Breuer Chapter 3: Markov Processes First hitting times

Example: M/M/c/c queue (contd.)

We wish to know the time until the first loss. Consider the Markovprocess Y with state space E = {0, . . . , c + 1} and generatormatrix

Q =

2 2 . . .

. . .. . .

c c 0 . . . . . . 0 0

Now

Z := inf{t 0 : Y t = c + 1}

is the random variable of the time until the first loss.

L. Breuer Chapter 3: Markov Processes First hitting times

Example: M/M/c/c queue (contd.)

We wish to know the time until the first loss. Consider the Markovprocess Y with state space E = {0, . . . , c + 1} and generatormatrix

Q =

2 2 . . .

. . .. . .

c c 0 . . . . . . 0 0

Now

Z := inf{t 0 : Y t = c + 1}

is the random variable of the time until the first loss.

L. Breuer Chapter 3: Markov Processes First hitting times

Phasetype distributions

Let Y = (Yt : t 0) denote an homogeneous Markov process withfinite state space {1, . . . ,m + 1}

and generator

Q =

(T 0 0

)where T is a square matrix of dimension m, a column vector and0 the zero row vector of the same dimension. The initialdistribution of Y shall be the row vector = (, m+1), with being a row vector of dimension m. The first states {1, . . . ,m}shall be transient, while the state m + 1 is absorbing. LetZ := inf{t 0 : Yt = m + 1} be the random variable of the timeuntil absorption in state m + 1.

L. Breuer Chapter 3: Markov Processes First hitting times

Phasetype distributions

Let Y = (Yt : t 0) denote an homogeneous Markov process withfinite state space {1, . . . ,m + 1} and generator

Q =

(T 0 0

)

where T is a square matrix of dimension m, a column vector and0 the zero row vector of the same dimension. The initialdistribution of Y shall be the row vector = (, m+1), with being a row vector of dimension m. The first states {1, . . . ,m}shall be transient, while the state m + 1 is absorbing. LetZ := inf{t 0 : Yt = m + 1} be the random variable of the timeuntil absorption in state m + 1.

L. Breuer Chapter 3: Markov Processes First hitting times

Phasetype distributions

Let Y = (Yt : t 0) denote an homogeneous Markov process withfinite state space {1, . . . ,m + 1} and generator

Q =

(T 0 0

)where T is a square matrix of dimension m,

a column vector and0 the zero row vector of the same dimension. The initialdistribution of Y shall be the row vector = (, m+1), with being a row vector of dimension m. The first states {1, . . . ,m}shall be transient, while the state m + 1 is absorbing. LetZ := inf{t 0 : Yt = m + 1} be the random variable of the timeuntil absorption in state m + 1.

L. Breuer Chapter 3: Markov Processes First hitting times

Phasetype distributions

Let Y = (Yt : t 0) denote an homogeneous Markov process withfinite state space {1, . . . ,m + 1} and generator

Q =

(T 0 0

)where T is a square matrix of dimension m, a column vector and0 the zero row vector of the same dimension.

The initialdistribution of Y shall be the row vector = (, m+1), with being a row vector of dimension m. The first states {1, . . . ,m}shall be transient, while the state m + 1 is absorbing. LetZ := inf{t 0 : Yt = m + 1} be the random variable of the timeuntil absorption in state m + 1.

L. Breuer Chapter 3: Markov Processes First hitting times

Phasetype distributions

Q =

(T 0 0

)where T is a square matrix of dimension m, a column vector and0 the zero row vector of the same dimension. The initialdistribution of Y shall be the row vector = (, m+1),

with being a row vector of dimension m. The first states {1, . . . ,m}shall be transient, while the state m + 1 is absorbing. LetZ := inf{t 0 : Yt = m + 1} be the random variable of the timeuntil absorption in state m + 1.

L. Breuer Chapter 3: Markov Processes First hitting times

Phasetype distributions

Q =

(T 0 0

)where T is a square matrix of dimension m, a column vector and0 the zero row vector of the same dimension. The initialdistribution of Y shall be the row vector = (, m+1), with being a row vector of dimension m.

The first states {1, . . . ,m}shall be transient, while the state m + 1 is absorbing. LetZ := inf{t 0 : Yt = m + 1} be the random variable of the timeuntil absorption in state m + 1.

L. Breuer Chapter 3: Markov Processes First hitting times

Phasetype dist