Stochastic Processes - Home | Department of adembo/math-136/nnotes.pdfStochastic Processes...

Click here to load reader

  • date post

  • Category


  • view

  • download


Embed Size (px)

Transcript of Stochastic Processes - Home | Department of adembo/math-136/nnotes.pdfStochastic Processes...

  • Stochastic Processes

    Amir Dembo (revised by Kevin Ross)

    August 21, 2013

    E-mail address : [email protected]

    Department of Statistics, Stanford University, Stanford, CA 94305.

  • Contents

    Preface 5

    Chapter 1. Probability, measure and integration 71.1. Probability spaces and -fields 71.2. Random variables and their expectation 101.3. Convergence of random variables 191.4. Independence, weak convergence and uniform integrability 25

    Chapter 2. Conditional expectation and Hilbert spaces 352.1. Conditional expectation: existence and uniqueness 352.2. Hilbert spaces 392.3. Properties of the conditional expectation 432.4. Regular conditional probability 46

    Chapter 3. Stochastic Processes: general theory 493.1. Definition, distribution and versions 493.2. Characteristic functions, Gaussian variables and processes 553.3. Sample path continuity 62

    Chapter 4. Martingales and stopping times 674.1. Discrete time martingales and filtrations 674.2. Continuous time martingales and right continuous filtrations 734.3. Stopping times and the optional stopping theorem 764.4. Martingale representations and inequalities 824.5. Martingale convergence theorems 884.6. Branching processes: extinction probabilities 90

    Chapter 5. The Brownian motion 955.1. Brownian motion: definition and construction 955.2. The reflection principle and Brownian hitting times 1015.3. Smoothness and variation of the Brownian sample path 103

    Chapter 6. Markov, Poisson and Jump processes 1116.1. Markov chains and processes 1116.2. Poisson process, Exponential inter-arrivals and order statistics 1196.3. Markov jump processes, compound Poisson processes 125

    Bibliography 127

    Index 129


  • Preface

    These are the lecture notes for a one quarter graduate course in Stochastic Pro-cesses that I taught at Stanford University in 2002 and 2003. This course is intendedfor incoming master students in Stanfords Financial Mathematics program, for ad-vanced undergraduates majoring in mathematics and for graduate students fromEngineering, Economics, Statistics or the Business school. One purpose of this textis to prepare students to a rigorous study of Stochastic Differential Equations. Morebroadly, its goal is to help the reader understand the basic concepts of measure the-ory that are relevant to the mathematical theory of probability and how they applyto the rigorous construction of the most fundamental classes of stochastic processes.

    Towards this goal, we introduce in Chapter 1 the relevant elements from measureand integration theory, namely, the probability space and the -fields of eventsin it, random variables viewed as measurable functions, their expectation as thecorresponding Lebesgue integral, independence, distribution and various notions ofconvergence. This is supplemented in Chapter 2 by the study of the conditionalexpectation, viewed as a random variable defined via the theory of orthogonalprojections in Hilbert spaces.

    After this exploration of the foundations of Probability Theory, we turn in Chapter3 to the general theory of Stochastic Processes, with an eye towards processesindexed by continuous time parameter such as the Brownian motion of Chapter5 and the Markov jump processes of Chapter 6. Having this in mind, Chapter3 is about the finite dimensional distributions and their relation to sample pathcontinuity. Along the way we also introduce the concepts of stationary and Gaussianstochastic processes.

    Chapter 4 deals with filtrations, the mathematical notion of information pro-gression in time, and with the associated collection of stochastic processes calledmartingales. We treat both discrete and continuous time settings, emphasizing theimportance of right-continuity of the sample path and filtration in the latter case.Martingale representations are explored, as well as maximal inequalities, conver-gence theorems and applications to the study of stopping times and to extinctionof branching processes.

    Chapter 5 provides an introduction to the beautiful theory of the Brownian mo-tion. It is rigorously constructed here via Hilbert space theory and shown to be aGaussian martingale process of stationary independent increments, with continuoussample path and possessing the strong Markov property. Few of the many explicitcomputations known for this process are also demonstrated, mostly in the contextof hitting times, running maxima and sample path smoothness and regularity.



    Chapter 6 provides a brief introduction to the theory of Markov chains and pro-cesses, a vast subject at the core of probability theory, to which many text booksare devoted. We illustrate some of the interesting mathematical properties of suchprocesses by examining the special case of the Poisson process, and more generally,that of Markov jump processes.

    As clear from the preceding, it normally takes more than a year to cover the scopeof this text. Even more so, given that the intended audience for this course has onlyminimal prior exposure to stochastic processes (beyond the usual elementary prob-ability class covering only discrete settings and variables with probability densityfunction). While students are assumed to have taken a real analysis class dealingwith Riemann integration, no prior knowledge of measure theory is assumed here.The unusual solution to this set of constraints is to provide rigorous definitions,examples and theorem statements, while forgoing the proofs of all but the mosteasy derivations. At this somewhat superficial level, one can cover everything in aone semester course of forty lecture hours (and if one has highly motivated studentssuch as I had in Stanford, even a one quarter course of thirty lecture hours mightwork).

    In preparing this text I was much influenced by Zakais unpublished lecture notes[Zak]. Revised and expanded by Shwartz and Zeitouni it is used to this day forteaching Electrical Engineering Phd students at the Technion, Israel. A secondsource for this text is Breimans [Bre92], which was the intended text book for myclass in 2002, till I realized it would not do given the preceding constraints. Theresulting text is thus a mixture of these influencing factors with some digressionsand additions of my own.

    I thank my students out of whose work this text materialized. Most notably Ithank Nageeb Ali, Ajar Ashyrkulova, Alessia Falsarone and Che-Lin Su who wrotethe first draft out of notes taken in class, Barney Hartman-Glaser, Michael He,Chin-Lum Kwa and Chee-Hau Tan who used their own class notes a year later ina major revision, reorganization and expansion of this draft, and Gary Huang andMary Tian who helped me with the intricacies of LATEX.

    I am much indebted to my colleague Kevin Ross for providing many of the exercisesand all the figures in this text. Kevins detailed feedback on an earlier draft of thesenotes has also been extremely helpful in improving the presentation of many keyconcepts.

    Amir Dembo

    Stanford, CaliforniaJanuary 2008


    Probability, measure and integration

    This chapter is devoted to the mathematical foundations of probability theory.Section 1.1 introduces the basic measure theory framework, namely, the proba-bility space and the -fields of events in it. The next building block are randomvariables, introduced in Section 1.2 as measurable functions 7 X(). This allowsus to define the important concept of expectation as the corresponding Lebesgueintegral, extending the horizon of our discussion beyond the special functions andvariables with density, to which elementary probability theory is limited. As muchof probability theory is about asymptotics, Section 1.3 deals with various notionsof convergence of random variables and the relations between them. Section 1.4concludes the chapter by considering independence and distribution, the two funda-mental aspects that differentiate probability from (general) measure theory, as wellas the related and highly useful technical tools of weak convergence and uniformintegrability.

    1.1. Probability spaces and -fields

    We shall define here the probability space (,F ,P) using the terminology of mea-sure theory. The sample space is a set of all possible outcomes of somerandom experiment or phenomenon. Probabilities are assigned by a set functionA 7 P(A) to A in a subset F of all possible sets of outcomes. The event space Frepresents both the amount of information available as a result of the experimentconducted and the collection of all events of possible interest to us. A pleasantmathematical framework results by imposing on F the structural conditions of a-field, as done in Subsection 1.1.1. The most common and useful choices for this-field are then explored in Subsection 1.1.2.

    1.1.1. The probability space (,F , P). We use 2 to denote the set of allpossible subsets of . The event space is thus a subset F of 2, consisting of allallowed events, that is, those events to which we shall assign probabilities. We nextdefine the structural conditions imposed on F .

    Definition 1.1.1. We say that F 2 is a -field (or a -algebra), if(a) F ,(b) If A F then Ac F as well (where Ac = \A).(c) If Ai F for i = 1, 2 . . . then also

    i=1 Ai F .

    Remark. Using DeMorgans law you can easily check that if Ai F for i = 1, 2 . . .and F is a -field, then also iAi F . Similarly, you can show that a -field isclosed under countably many elementary set operations.



    Definition 1.1.2. A pair (,F) with F a -field of subsets of is called ameasurable space. Given a measurable space, a probability measure P is a functionP : F [0, 1], having the following properties:(a) 0 P(A) 1 for all A F .(b) P() = 1.(c) (Countable additivity) P(A) =

    n=1 P(An) whenever