Probability Theory and Mathematical StatisticsLecture 11: Special Probability Densities

Chih-Yuan Hung

School of Economics and ManagementDongguan University of Technology

May 22, 2019

Joseph Hung Probability May 22, 2019 1 / 26

Introduction

This chapter focus on the most prominent probability densities instatistics

We study their parameters

e.g.: mean, µ; variance, σ2

Obtain these parameters by either integration directly or by momentgenerating functions

some density functions are important in itself others are important instudying the technique of solving the parameters

Joseph Hung Probability May 22, 2019 2 / 26

Definition (1. Uniform Distribution)

A random variable X has a uniform distribution and it is referred to as acontinuous uniform distribution random variable if and only if itsprobability density is given by

u(x ; α, β) =

{1

β−α for α < x < β

0 elsewhere

The parameters α and β of this probability density are real constants withα < β

Joseph Hung Probability May 22, 2019 3 / 26

Theorem (1)

The mean and variance of the uniform distribution are given by

µ =α + β

2and σ2 =

1

12(β− α)2

Proof.

By definition,

u(x ; α, β) =1

β− αfor α < x < β

µ = E (X ) =∫ β

αx

1

β− αdx

=β2 − α2

2(β− α)

=β + α

2

Joseph Hung Probability May 22, 2019 4 / 26

Proof.

E (X 2) =∫ β

αx2

1

β− αdx

=β3 − α3

3(β− α)

=β2 + αβ + α2

3

σ2 =E (X 2)− [E (X )]2

=β2 + αβ + α2

3− β2 + 2αβ + α2

4

=β2 − 2αβ + α2

12

=1

12(β− α)2

Joseph Hung Probability May 22, 2019 5 / 26

Moment Generation Function

Mx (t) = E (etx ) =∫ β

αetx

1

β− αdx

=eβt − eαt

t(β− α)

Joseph Hung Probability May 22, 2019 6 / 26

Application: First Price Sealed Bid Auction

Suppose two people, 1 and 2, compete for an object in a first pricesealed bid auction.

They have the private value for the object, vi for i = 1, 2 , and vi isuniformly distributed between (0, 1).

According to their bid, b1, b2, 1’s payoffs is

π1(b1, b2) =

{v1 − b1 if b1 > b2

0 if b1 < b2

Assume bi (0) = 0

What is the (monotonically increasing) symmetric equilibrium biddingstrategy? (b1 = b2 = b) with the same value)

Joseph Hung Probability May 22, 2019 7 / 26

The solution is to solve

maxb1

P(b2(v2) < b1)(v1 − b1)

By monotonically increasing assumption v2 = b−12 (b2). So

maxb1

P(v2 < b−12 (b1))(v1 − b1)

=maxb1

F (b−12 (b1))(v1 − b1)

=maxb1

b−12 (b1)(v1 − b1)

Differentiating above equation and let it to zero,

(v1 − b1)b−1′2 (b1) = b−12 (b1)

Joseph Hung Probability May 22, 2019 8 / 26

We can rewrite the first order condition as

v1 − b1 = b′2(v1)v1

Again by the symmetry of equilibrium assumption,b1(v1) = b2(v1) = b(v1)

b′(v1) · v1 + b(v1) = v1

ord

dv1[v1b1(v1)] = v1

We have

v1b(v1) =∫ 1

0v1dv1

b(v1) =v12+ c

Since b(0) = 0, b(v1) =v12 .

Joseph Hung Probability May 22, 2019 9 / 26

Gamma Function

We have dealt with random variables with the following form

f (x) =

{kxα−1e

−xβ for x > 0

0 elsewhere

where α > 0, β > 0, and k is determined by∫ ∞

0kxα−1e−x/βdx = 1

Define gamma function as

Γ(α) =∫ ∞

0y α−1e−ydy

for α > 0. Gamma function satisfies the recursion formula

Γ(α) = (α− 1)Γ(α− 1)

Joseph Hung Probability May 22, 2019 10 / 26

Gamma Distribution

Knowing that Γ(1) =∫ ∞0 e−ydy = 1, Γ(α) = (α− 1)!.

So, let y = xβ , ∫ ∞

0kxα−1e−x/βdx = kβΓ(α) = 1

k =1

β(α− 1)!

Definition (Gamma Distribution)

A random variable X has a gamma distribution and it is referred to as agamma random variable if and only if its probability density is given by

g(x ; α, β) =

{1

β(α−1)!xα−1e

−xβ for x > 0

0 elsewhere

where α > 0, β > 0.

Joseph Hung Probability May 22, 2019 11 / 26

Gamma Distribution

If α is not integer, we need a table to find out the special cases.

The simplest case of gamma distribution is when α = 1 and β = θ.

Joseph Hung Probability May 22, 2019 12 / 26

Definition

A random variable X has an exponential distribution and it is referredto as an exponential random variable if and only if its probability density isgiven by

g(x ; θ) =

{1βe

−xθ for x > 0

0 elsewhere

where θ > 0

Joseph Hung Probability May 22, 2019 13 / 26

Consider there is the probability of getting x successes during a timeinterval of length t

α · ∆t the probability to success in a very small time interval

The probability is not depends on time t

The number of success (in a given time) is a Poisson random variablewith λ = αt

What is the probability of waiting time? (until y)

F (y) =P(Y ≤ y) = 1− P(Y > y)

=1− P(0 success in a time interval of length y)

=1− p(0; αy)

=1− (αy)0e−αy

0!=1− e−αy

Joseph Hung Probability May 22, 2019 14 / 26

Differentiate F (y), we have

f (y) =

{αe−αy for y > 0

0 elsewhere

which is an exponential distribution with θ = 1α

Example (1)

At a certain location on highway I-10, the number of cars exceeding thespeed limit by more than 10 miles per hour in half an hour is a randomvariable having a Poisson distribution with λ = 8.4. What is theprobability of a waiting time of less than 5 minutes between cars exceedingthe speed limit by more than 10 miles per hour?

Using half an hour as the unit of time, we have α = λ = 8.4,This can be formulated as exponential distribution with θ = 1

8.4 .

P

(x <

1

6

)=∫ 1

6

08.4e−8.4xdx = −e−1.4 + 1

Joseph Hung Probability May 22, 2019 15 / 26

Chi-square Distribution

Another special case of gamma distribution arises when α = v2 and β = 2.

Definition (Chi-square Distribution)

A random variable X has a chi-square distribution and it is referred to asa chi-square random variable if and only if its probability density is given by

f (x , v) =

{1

2v/2 Γ(v/2)xv−22 e−

12 for x > 0

0 elsewhere

Joseph Hung Probability May 22, 2019 16 / 26

Mean and Variance of Gamma Distribution

Theorem

The rth moment about the origin of the gamma distribution is given by

µ′r =βrΓ(α + r)

Γ(α)

Proof.

By definition

µ′r =∫ ∞

0x r

1

βαΓ(α)xα−1e−x/βdx

=βr

Γ(α)·∫ ∞

0y α+r−1e−ydy

=βr

Γ(α)Γ(α + r − 1)

Joseph Hung Probability May 22, 2019 17 / 26

Theorem

The mean and the variance of the gamma distribution are given by

µ = αβ and σ2 = αβ2

Proof.

By the theorem above

µ = µ′1 =β1Γ(α + 1)

Γ(α)= αβ

and

µ′2 =β2Γ(α + 2)

Γ(α)= α(α + 1)β2

Soσ2 = µ′2 − µ2 = αβ2

Joseph Hung Probability May 22, 2019 18 / 26

Mean and Variance of Exponential and Chi-square

The mean and the variance of the exponential distribution(α = 1, β = θ) are given by

µ = θ and σ2 = θ2

The mean and the variance of the chi-square distribution(α = v/2, β = 2) are given by

µ = v and σ2 = 2v

The moment generating function of gamma distribution is

Mx (t) = (1− βt)−α

Joseph Hung Probability May 22, 2019 19 / 26

The uniform random variable is a special case of the beta distribution.

Definition

A random variable X has an beta distribution and it is referred to as anbeta random variable if and only if its probability density is given by

f (x ; α, β) =

{Γ(α+β)

Γ(α)Γ(β)xα−1(1− x)β−1 for 0 < x < 1

0 elsewhere

where α > 0 and β > 0

The beta distribution has found important applications in Bayesianinferencewhich means we want a flexible parameter in binomial probability tosuccess, θ.With the property ∫ 1

0

Γ(α + β)

Γ(α)Γ(β)xα−1(1− x)β−1dx = 1

Joseph Hung Probability May 22, 2019 20 / 26

Define the integral∫ 10 xα−1(1− x)β−1dx = Γ(α)Γ(β)

Γ(α+β)as beta function,

B(α, β)

Theorem

The mean and the variance of the beta distribution are given by

µ =α

α + βand σ2 =

αβ

(α + β)2(α + β + 1)

Joseph Hung Probability May 22, 2019 21 / 26

Proof.

By definition,

µ =Γ(α + β)

Γ(α)Γ(β)·∫ 1

0x · xα−1(1− x)β−1dx

=Γ(α + β)

Γ(α)Γ(β)· Γ(α + 1)Γ(β)

Γ(α + β + 1)

=α

α + β

Similarly, µ′2 =(α+1)α

(α+β+1)(α+β)So,

σ2 = µ′2 − µ2 =(α + 1)α

(α + β + 1)(α + β)−(

α

α + β

)2

=αβ

(α + β)2(α + β + 1)

Joseph Hung Probability May 22, 2019 22 / 26

Normal Distribution

The normal distribution, which we shall study in this section, is inmany ways the cornerstone of modern statistical theory.

Observed an astonishing degree of regularity in errors of measurement.

The mathematical properties of such normal curves were first studiedby Abraham de Moivre (1667-1745), Pierre Laplace (1749-1827), andKarl Gauss (1777-1855).

Definition

A random variable X has a normal distribution and it is referred to as anormal random variable if and only if its probability density is given by

n(x ; µ, σ) =1

σ√

2πe−

12 (

x−µσ )2 for −∞ < x < ∞

where σ > 0

Joseph Hung Probability May 22, 2019 23 / 26

Normal Distribution

Joseph Hung Probability May 22, 2019 24 / 26

First, let z = x−µσ

Joseph Hung Probability May 22, 2019 25 / 26

Homework

Work with your partner (in group)

hand in the homework to the editor group on duty before 17:00,Sunday.

Group editor on duty shall organize the final answers and send the fileof final answer to 1307455914@qq.com before next Tuesday

HW: Chapter 5- 41, 43, 51, 57, 63, 69, 75, 81

Joseph Hung Probability May 22, 2019 26 / 26

Questions??

Joseph Hung Probability May 22, 2019 27 / 26