CIS 2033 based onDekking et al. A Modern Introduction to Probability and Statistics. 2007

Slides by Michael Maurizi

Instructor Longin Jan Latecki

C9: Joint Distributions and Independence

9.1 – Joint Distributions of Discrete Random Variables

Joint Distribution: the combined distribution of two or more random variables defined on the same sample space Ω

Joint Distribution of two discrete random variables:The joint distribution of two discrete random variables X and Y can be obtained by using the probabilities of all possible values of the pair (X,Y)

bababapbap

p

XY ,for)Y,P(X),(),(

[0,1]R : 2

Joint Probability Mass function p of two discrete random variables X and Y:

Joint Distribution function F of two random variables X and Y: Can be thought of as the sum of the elements in box it makes with the upper-left corner.

bababaF

p

,for)Y,P(X),(

[0,1]R : 2

9.1 – Joint Distributions of Discrete Random Variables

Marginal Distribution: Obtained by adding up the rows or columns of a joint probability mass function table. Literally written in the margins.

Let p(a,b) be a joint pmf of RVs S and M. The marginal pmfs are then given by

Example: Joint Distribution of S and M. S = The sum of two dice, M = The maximum of two dice.

bpS(b)a 1 2 3 4 5 6

2 1/36

0 0 0 0 0 1/36

3 0 2/36

0 0 0 0 2/36

4 0 1/36

2/36

0 0 0 3/36

5 0 0 2/36

2/36

0 0 4/36

6 0 0 1/36

2/36

2/36

0 5/36

7 0 0 0 2/36

2/36

2/36 6/36

8 0 0 0 1/36

2/36

2/36 5/36

9 0 0 0 0 2/36

2/36 4/36

10 0 0 0 0 1/36

2/36 3/36

11 0 0 0 0 0 2/36 2/36

12 0 0 0 0 0 1/36 1/36

pM(a)

1/36

3/36

5/36

7/36

9/36

11/36

1

b

S bapapaSP ),()()(

a

M bapbpbMP ),()()(

9.2 – Joint Distributions of Continuous Random Variables Joint Continuous Distribution: Like an ordinary continuous

random variable, only works for a range of values. There must exist a function f that fulfills the following properties for there to be a joint continuous distribution:

1),(

and allfor 0),(

and allfor ),()Y,X(P

RR:

22112211

2

1

1

2

2

dxdyyxf

yxyxf

babadxdyyxfbaba

fb

a

b

a

),(lim),()X(P)( baFaFaaFb

X

Marginal distribution function of X:

),(lim),()Y(P)( baFbFbbFa

Y

Marginal distribution function of Y:

9.2 – Joint Distributions of Continuous Random Variables

Joint distribution function:F(a,b) can be constructed given f(x,y), and vice versa

),(and),(),(2

yxFyx

f(x,y)dxdyyxfbaFa b

Marginal probability density function:You need to integrate out the unwanted random variable to get the marginal distribution.

dxyxfyfdyyxfxf YX ),()(and),()(

9.3 – More than Two Random VariablesAssuming we have n random variables X1, X2, X3, … Xn. We can get the joint distribution function and the joint probability mass functions.

)X,,X,X(P),,,(

:function massy probabilitJoint

)X,,X,X(P),,,(

:functionon distributiJoint

,,,for

n221121

n221121

21

nn

nn

n

aaaaaap

aaaaaaF

aaa

9.4 – Independent Random Variables

Tests for Independence: Two random variables X and Y are independent if and only if every event involving X is independent of every event involving Y.

This also applies to joint distributions using more than two random variables.

)()(),(

or

)P(Y)P(X)Y,P(X

or

)P(Y)P(X)Y,P(X

and possible allfor

bFaFbaF

baba

baba

ba

YX

9.5 – Propagation of Independence

Independence after a change of variable: If a function is applied to several independent random variables, the new resulting random variables will also be independent.

Example 3. 6, p. 48, in Baron book