Chapter 3: Markov Processes First hitting times · PDF fileChapter 3: Markov Processes First...

Click here to load reader

  • date post

    18-Jun-2018
  • Category

    Documents

  • view

    222
  • download

    0

Embed Size (px)

Transcript of Chapter 3: Markov Processes First hitting times · PDF fileChapter 3: Markov Processes First...

  • Chapter 3: Markov ProcessesFirst hitting times

    L. BreuerUniversity of Kent, UK

    November 3, 2010

    L. Breuer Chapter 3: Markov Processes First hitting times

  • Example: M/M/c/c queue

    Arrivals: Poisson process with rate > 0

    Service times: exponentially distributed with rate > 0c serversno waiting roomIf upon an arrival the system is filled, i.e. all servers are busy, thisarriving user is not admitted into the system. In this case we saythat the arriving user is lost. Queueing systems with the possibilityof such an event are thus called loss systems.

    L. Breuer Chapter 3: Markov Processes First hitting times

  • Example: M/M/c/c queue

    Arrivals: Poisson process with rate > 0Service times: exponentially distributed with rate > 0

    c serversno waiting roomIf upon an arrival the system is filled, i.e. all servers are busy, thisarriving user is not admitted into the system. In this case we saythat the arriving user is lost. Queueing systems with the possibilityof such an event are thus called loss systems.

    L. Breuer Chapter 3: Markov Processes First hitting times

  • Example: M/M/c/c queue

    Arrivals: Poisson process with rate > 0Service times: exponentially distributed with rate > 0c servers

    no waiting roomIf upon an arrival the system is filled, i.e. all servers are busy, thisarriving user is not admitted into the system. In this case we saythat the arriving user is lost. Queueing systems with the possibilityof such an event are thus called loss systems.

    L. Breuer Chapter 3: Markov Processes First hitting times

  • Example: M/M/c/c queue

    Arrivals: Poisson process with rate > 0Service times: exponentially distributed with rate > 0c serversno waiting room

    If upon an arrival the system is filled, i.e. all servers are busy, thisarriving user is not admitted into the system. In this case we saythat the arriving user is lost. Queueing systems with the possibilityof such an event are thus called loss systems.

    L. Breuer Chapter 3: Markov Processes First hitting times

  • Example: M/M/c/c queue

    Arrivals: Poisson process with rate > 0Service times: exponentially distributed with rate > 0c serversno waiting roomIf upon an arrival the system is filled, i.e. all servers are busy, thisarriving user is not admitted into the system.

    In this case we saythat the arriving user is lost. Queueing systems with the possibilityof such an event are thus called loss systems.

    L. Breuer Chapter 3: Markov Processes First hitting times

  • Example: M/M/c/c queue

    Arrivals: Poisson process with rate > 0Service times: exponentially distributed with rate > 0c serversno waiting roomIf upon an arrival the system is filled, i.e. all servers are busy, thisarriving user is not admitted into the system. In this case we saythat the arriving user is lost.

    Queueing systems with the possibilityof such an event are thus called loss systems.

    L. Breuer Chapter 3: Markov Processes First hitting times

  • Example: M/M/c/c queue

    Arrivals: Poisson process with rate > 0Service times: exponentially distributed with rate > 0c serversno waiting roomIf upon an arrival the system is filled, i.e. all servers are busy, thisarriving user is not admitted into the system. In this case we saythat the arriving user is lost. Queueing systems with the possibilityof such an event are thus called loss systems.

    L. Breuer Chapter 3: Markov Processes First hitting times

  • Example: M/M/c/c queue (contd.)

    The queue described above is a skip-free Markov process Y withstate space E = {0, . . . , c}

    and generator matrix

    Q =

    2 2 . . .

    . . .. . .

    (c 1) (c 1) c c

    up to the first loss.

    L. Breuer Chapter 3: Markov Processes First hitting times

  • Example: M/M/c/c queue (contd.)

    The queue described above is a skip-free Markov process Y withstate space E = {0, . . . , c} and generator matrix

    Q =

    2 2 . . .

    . . .. . .

    (c 1) (c 1) c c

    up to the first loss.

    L. Breuer Chapter 3: Markov Processes First hitting times

  • Example: M/M/c/c queue (contd.)

    We wish to know the time until the first loss.

    Consider the Markovprocess Y with state space E = {0, . . . , c + 1} and generatormatrix

    Q =

    2 2 . . .

    . . .. . .

    c c 0 . . . . . . 0 0

    Now

    Z := inf{t 0 : Y t = c + 1}

    is the random variable of the time until the first loss.

    L. Breuer Chapter 3: Markov Processes First hitting times

  • Example: M/M/c/c queue (contd.)

    We wish to know the time until the first loss. Consider the Markovprocess Y with state space E = {0, . . . , c + 1}

    and generatormatrix

    Q =

    2 2 . . .

    . . .. . .

    c c 0 . . . . . . 0 0

    Now

    Z := inf{t 0 : Y t = c + 1}

    is the random variable of the time until the first loss.

    L. Breuer Chapter 3: Markov Processes First hitting times

  • Example: M/M/c/c queue (contd.)

    We wish to know the time until the first loss. Consider the Markovprocess Y with state space E = {0, . . . , c + 1} and generatormatrix

    Q =

    2 2 . . .

    . . .. . .

    c c 0 . . . . . . 0 0

    Now

    Z := inf{t 0 : Y t = c + 1}

    is the random variable of the time until the first loss.

    L. Breuer Chapter 3: Markov Processes First hitting times

  • Example: M/M/c/c queue (contd.)

    We wish to know the time until the first loss. Consider the Markovprocess Y with state space E = {0, . . . , c + 1} and generatormatrix

    Q =

    2 2 . . .

    . . .. . .

    c c 0 . . . . . . 0 0

    Now

    Z := inf{t 0 : Y t = c + 1}

    is the random variable of the time until the first loss.

    L. Breuer Chapter 3: Markov Processes First hitting times

  • Phasetype distributions

    Let Y = (Yt : t 0) denote an homogeneous Markov process withfinite state space {1, . . . ,m + 1}

    and generator

    Q =

    (T 0 0

    )where T is a square matrix of dimension m, a column vector and0 the zero row vector of the same dimension. The initialdistribution of Y shall be the row vector = (, m+1), with being a row vector of dimension m. The first states {1, . . . ,m}shall be transient, while the state m + 1 is absorbing. LetZ := inf{t 0 : Yt = m + 1} be the random variable of the timeuntil absorption in state m + 1.

    L. Breuer Chapter 3: Markov Processes First hitting times

  • Phasetype distributions

    Let Y = (Yt : t 0) denote an homogeneous Markov process withfinite state space {1, . . . ,m + 1} and generator

    Q =

    (T 0 0

    )

    where T is a square matrix of dimension m, a column vector and0 the zero row vector of the same dimension. The initialdistribution of Y shall be the row vector = (, m+1), with being a row vector of dimension m. The first states {1, . . . ,m}shall be transient, while the state m + 1 is absorbing. LetZ := inf{t 0 : Yt = m + 1} be the random variable of the timeuntil absorption in state m + 1.

    L. Breuer Chapter 3: Markov Processes First hitting times

  • Phasetype distributions

    Let Y = (Yt : t 0) denote an homogeneous Markov process withfinite state space {1, . . . ,m + 1} and generator

    Q =

    (T 0 0

    )where T is a square matrix of dimension m,

    a column vector and0 the zero row vector of the same dimension. The initialdistribution of Y shall be the row vector = (, m+1), with being a row vector of dimension m. The first states {1, . . . ,m}shall be transient, while the state m + 1 is absorbing. LetZ := inf{t 0 : Yt = m + 1} be the random variable of the timeuntil absorption in state m + 1.

    L. Breuer Chapter 3: Markov Processes First hitting times

  • Phasetype distributions

    Let Y = (Yt : t 0) denote an homogeneous Markov process withfinite state space {1, . . . ,m + 1} and generator

    Q =

    (T 0 0

    )where T is a square matrix of dimension m, a column vector and0 the zero row vector of the same dimension.

    The initialdistribution of Y shall be the row vector = (, m+1), with being a row vector of dimension m. The first states {1, . . . ,m}shall be transient, while the state m + 1 is absorbing. LetZ := inf{t 0 : Yt = m + 1} be the random variable of the timeuntil absorption in state m + 1.

    L. Breuer Chapter 3: Markov Processes First hitting times

  • Phasetype distributions

    Let Y = (Yt : t 0) denote an homogeneous Markov process withfinite state space {1, . . . ,m + 1} and generator

    Q =

    (T 0 0

    )where T is a square matrix of dimension m, a column vector and0 the zero row vector of the same dimension. The initialdistribution of Y shall be the row vector = (, m+1),

    with being a row vector of dimension m. The first states {1, . . . ,m}shall be transient, while the state m + 1 is absorbing. LetZ := inf{t 0 : Yt = m + 1} be the random variable of the timeuntil absorption in state m + 1.

    L. Breuer Chapter 3: Markov Processes First hitting times

  • Phasetype distributions

    Let Y = (Yt : t 0) denote an homogeneous Markov process withfinite state space {1, . . . ,m + 1} and generator

    Q =

    (T 0 0

    )where T is a square matrix of dimension m, a column vector and0 the zero row vector of the same dimension. The initialdistribution of Y shall be the row vector = (, m+1), with being a row vector of dimension m.

    The first states {1, . . . ,m}shall be transient, while the state m + 1 is absorbing. LetZ := inf{t 0 : Yt = m + 1} be the random variable of the timeuntil absorption in state m + 1.

    L. Breuer Chapter 3: Markov Processes First hitting times

  • Phasetype dist