Intro probability 2

Click here to load reader

  • date post

    11-May-2015
  • Category

    Documents

  • view

    2.765
  • download

    1

Embed Size (px)

Transcript of Intro probability 2

  • 1.Probability Theory Random Variables Phong VO vdphong@fit.hcmus.edu.vnSeptember 11, 2010 Typeset by FoilTEX

2. Random Variables Denition 1. A random variable is a mapping X : S R that associates a unique numerical value X() to each outcome . Letting X denote the random variable that is dened as the sum of two fair dice, then 1P {X = 2) = P ({(1, 1))) = ,362P {X = 3) = P ({(1, 2), (2, 1))) =, 36 3P {X = 4) = P ({(1, 3), (2, 2), (3, 1))) = 36 Typeset by FoilTEX 1 3. Distribution Functions and Probability FunctionsDenition 2. The cumulative distribution function CDF FX : R [0, 1] of a r.v X is dened by FX (x) = P (X x). Example 1. Flip a fair coin twice and let X be the number of heads. Then P (X = 0) = P (X = 2) = 1/4 and P (X = 1) = 1/2. The distribution function is Typeset by FoilTEX 2 4. 0 x 0, i p(i) = P (X = i) = e, i = 0, 1, . . . i! Typeset by FoilTEX 9 11. Continuous Random VariablesDenition 4. A r.v X is is continuous if there exists a function fX such that fX (x) 0x, fX (x)dx = 1 and for every a b, b P (a < X < b) =fX (x)dxa The function fX is called the probability density function(PDF). We have thatxFX (x) =fX (t)dt and fX (x) = FX (x) at all points x at which FX is dierentiable. Typeset by FoilTEX 10 12. If X is continuous then P (X = x) = 0x f (x) is dierent from P (X = 0)inthecontinuouscase a PDF can be bigger than 1 (unlike a mass function) 15 x [0, 5 ]f (x) =0 o.w then f (x) 0 and f (x)dx = 1 so this is a well-dened PDF even though f (x) = 5 in some places. Typeset by FoilTEX 11 13. Lemma 1. Let F be the CDF for a r.v X. Then:1. P (X = x) = F (x) F (x) where F (x) = limyxF (y),2. P (x < X y) = F (y) F (x),3. P (X > x) = 1 F (x),4. If X is continuous thenP (a < X < b) = P (a X < b) = P (a < X b) = P (a X b) Typeset by FoilTEX 12 14. The Uniform Random Variable An random variable is said to be uniformly distributed over the interval (0, 1) if its probability density function is given by1, 0x1 f (x) =0, otherwise In general case, 1 ,xf (x) = 0, otherwise Typeset by FoilTEX 13 15. Example 4. Calculate the cumulative distribution function of a random variable uniformly distributed over (, ). Typeset by FoilTEX 14 16. Exponential Random Variables A continuous random variable whose probability density function is given, for some > 0, by ex, if x 0 f (x) = 0,if x 0 is said to be an exponential random variable with parameter . Typeset by FoilTEX 15 17. Gamma Random VariablesA continuous random variable whose density is given by ex (x)1() ,if x 0f (x) =0,if x 0 for some > 0, > 0 is said to be a gamma random variable with parameter , . The quantity () is called the gamma function and is dened by () =exx1dx 0 Typeset by FoilTEX 16 18. Normal Random VariablesX is a normal random variable with parameters (, 2) if the density of X is given by1(x)2 /2 2 f (x) = e x (3)2 Typeset by FoilTEX 17 19. Remarks Read X F as X has distribution F . X is a r.v; x denotes a particular value of the r.v; n and p (i.e Binomial distribution) are parameters, that is, xed real numbers. Parameters is usually unknown and must be estimated from data. In practice, we think of r.v like a random number but formally it is a mapping dened on some sample space. Typeset by FoilTEX 18 20. Jointly Distributed Random Variables Given a pair of discrete r.vs X and Y , dene the joint mass function by f (x, y) = P (X = x, Y = y).Denition 5. In the continuous case, we call a function f (x, y) a pdf for the r.vs (X, Y ) if1. f (x, y) 0 (x, y), 2. f (x, y)dxdy = 1 and, for any set A R R, P ((X, Y ) A) =Af (x, y)dxdy. In the discrete or continuous case we dene the joint CDF as FX,Y (x, y) = P (X x, Y y). Typeset by FoilTEX 19 21. Example 5. At a party N men throw their hats into the center of a room. The hats are mixed up and each man randomly selects one. Find the expected number of men that select their own hats.Example 6. Suppose there are 25 dierent types of coupons and suppose that each time one obtains a coupon, it is equally likely to be any one of the 25 types. Compute the expected number of dierent types that are contained in a set of 10 coupons. Typeset by FoilTEX 20 22. Marginal Distributions Denition 6. If (X, Y ) have a joint distribution with mass function fX,Y , then the marginal mass function for X is dened by fX (x) = P (X = x) = P (X = x, Y = y) = f (x, y) yy and the marginal mass function for Y is dened byfY (y) = P (Y = y) = P (X = x, Y = y) = f (x, y) xx Typeset by FoilTEX 21 23. Example 7. Calculate the marginal distributions for X and Y from table below Y=0Y=1 X=0 1/10 2/10 3/10 X=1 3/10 4/10 7/10 4/10 6/10Denition 7. For continuous r.vs, the marginal densities arefX (x) =f (x, y)dy and fY (y) = f (x, y)dx The corresponding marginal distribution functions are denoted by FX and FY .Example 8. Suppose that Typeset by FoilTEX 22 24. x+y if 0 x 1, 0 y 1f (x, y) = 0 otherwise Then1 1 1 1fY (y) = (x + y)dx = xdx + ydx = + y. 0 0 0 2 Typeset by FoilTEX 23 25. Independent Random VariablesDenition 8. Two r.vs X and Y are said to be independent if, for every A and B, P (X A, Y B) = P (X A)P (Y B)Theorem 1. Let X and Y have joint pdf fX,Y . Then X and Y are independent is and only if fX,Y (x, y) = fX (x)fY (y) x, y.Example 9. Suppose that X and Y are independent and both have the same density Typeset by FoilTEX 24 26. 2x if 0 x 1 f (x) = 0 otherwise Let nd P (X + Y 1)?Theorem 2. Suppose that the range of X and Y is a rectangle (possibly innite). If f (x, y) = g(x)h(y) for some functions g and h (not necessarily probability density functions) then X and Y are independent.Example 10. Let X and Y have density 2e(x+2y) if x > 0 and y > 0f (x, y) = 0 otherwise. The range of X and Y is the rectangle (0, ) (0, ). We can write Typeset by FoilTEX 25 27. f (x, y) = g(x)h(y) where g(x) = 2ex and h(y) = e2y . Thus, X and Y are independent. Typeset by FoilTEX 26 28. Conditional Distributions One of the most useful concepts in probability theory We are often interested in calculating probabilities when some partial information is available Calculating a desired probability or expectation it is useful to rst condition on some appropriate r.v Denition 9. The redconditional probability mass function is P (X = x, Y = y) fX,Y (x, y)fX|Y (x|y) = P (X = x|Y = y) = = P (Y = y)fY (y) Typeset by FoilTEX 27 29. if fY (y) > 0.Denition 10. For continuous r.vs, the conditional probability density function is fX|Y (x|y) fX|Y (x|y) =fY (y) assuming that fY (y) > 0. Then,P (X A|Y = y) =fX|Y (x|y)dx. AExample 11. Suppose that X U nif (0, 1). After obtaining a value of X we generate Y |X = x U nif (x, 1). What is the marginal distribution of Y ? Typeset by FoilTEX 28 30. Multivariate Distributions and IID Samples Let call X(X1, . . . , Xn), where X1, . . . , Xn are r.vs, a random vector. If X1, . . . , Xn are independent and each has the same marginal distribution with density f , we say that X1, . . . , Xn are IID (independent and identically distributed). Much of statistical theory and practice begins with IID observations. Typeset by FoilTEX 29 31. Transformations of Random Variables Suppose that X is a r.v, Y = r(X) be a function of X, i.e. Y = X 2 or Y = ex. How do we compute the PDF and CDF of Y ? In the discrete casef Y (y) = P (Y = y) = P (r(X) = y) = P ({x; r(x) = y}) = P (X r1(y)) In the continuous case1. For each y, nd the set Ay = {x : r(x) y} Typeset by FoilTEX 30 32. 2. Find the CDFFY (y) = P (Y y) = P (r(X) y) (4) = P ({x; r(x) y}) =fX (x)dx (5)Ay3. The PDF is fY (y) = FY (y)x Example 12. Let fX (x) = ex for x > 0. Then FX (x) = 0 fX (s)ds = 1 ex. Let Y = r(X) = logX. Then Ay = {x : x ey } and yFY (y) = P (Y y) = P (logX y) = P (X ey ) = FX (ey ) = 1 ee . Typeset by FoilTEX 31 33. yTherefore, fY (y) = ey ee for y R. Typeset by FoilTEX 32 34. Transformations of Several Random Variables Typeset by FoilTEX 33