Distributions Basics of mathematical stats Confidence intervals

Introductory EconometricsSession 3 - Distribution and confidence intervals

Roland Rathelot

Sciences Po

July 2011

Rathelot

Introductory Econometrics

Distributions Basics of mathematical stats Confidence intervals

The Normal distribution

I If X is a r.v. normally distributed, X N(, 2),

fX (x) =1

2exp

[(x )

2

22

]I The pdf of the standard normal distribution is:

(z) =12

exp[z2/2

]

Rathelot

Introductory Econometrics

Distributions Basics of mathematical stats Confidence intervals

Some properties of the Normal distribution

I If X N(, 2) then (X )/ N(0, 1)I If X N(, 2) then aX + b N(a+ b, a22)I Any linear combination of independent, identically distributed

normal rv has a normal distribution

I X and Y are jointly normally distributed. In this case, if Xand Y are uncorrelated then they are independent

Rathelot

Introductory Econometrics

Distributions Basics of mathematical stats Confidence intervals

Some properties of the Normal distribution

I If X N(, 2) then (X )/ N(0, 1)I If X N(, 2) then aX + b N(a+ b, a22)I Any linear combination of independent, identically distributed

normal rv has a normal distribution

I X and Y are jointly normally distributed. In this case, if Xand Y are uncorrelated then they are independent

Rathelot

Introductory Econometrics

Distributions Basics of mathematical stats Confidence intervals

Some properties of the Normal distribution

I If X N(, 2) then (X )/ N(0, 1)I If X N(, 2) then aX + b N(a+ b, a22)I Any linear combination of independent, identically distributed

normal rv has a normal distribution

I X and Y are jointly normally distributed. In this case, if Xand Y are uncorrelated then they are independent

Rathelot

Introductory Econometrics

Distributions Basics of mathematical stats Confidence intervals

Some properties of the Normal distribution

I If X N(, 2) then (X )/ N(0, 1)I If X N(, 2) then aX + b N(a+ b, a22)I Any linear combination of independent, identically distributed

normal rv has a normal distribution

I X and Y are jointly normally distributed. In this case, if Xand Y are uncorrelated then they are independent

Rathelot

Introductory Econometrics

Distributions Basics of mathematical stats Confidence intervals

The chi-square distribution

I Let Zi , i = 1 . . .Q be Q independent rv distributed asstandard normal

I Then, the sum of the squares of the Zi , X , is known to have achi-square distribution with Q degrees of freedom (df)

X =Qi=1

Z 2i 2q

I Expectation and variance

E[X ] = Q;V [X ] = 2Q

Rathelot

Introductory Econometrics

Distributions Basics of mathematical stats Confidence intervals

The chi-square distribution

I Let Zi , i = 1 . . .Q be Q independent rv distributed asstandard normal

I Then, the sum of the squares of the Zi , X , is known to have achi-square distribution with Q degrees of freedom (df)

X =Qi=1

Z 2i 2q

I Expectation and variance

E[X ] = Q;V [X ] = 2Q

Rathelot

Introductory Econometrics

Distributions Basics of mathematical stats Confidence intervals

The t distribution

I Let Z have a standard normal distribution

I Let X have a chi-square distribution with Q degrees offreedom

I Assume Z and X are independent

I Then, T defined as

T =ZX/Q

has a t distribution with Q df

T tQ

Rathelot

Introductory Econometrics

Distributions Basics of mathematical stats Confidence intervals

The t and the Normal

I An approximate statement: The t is just a normal withthicker tails

I The pdf of the t and the normal look close: just the thicknessof the tails differ

I The lower the df of the t, the thicker the tails

I When the df tends to infinity, the t tends to a normal

Rathelot

Introductory Econometrics

Distributions Basics of mathematical stats Confidence intervals

The F distribution

I Let X1 have a chi-square distribution with Q1 degrees offreedom

I Let X2 have a chi-square distribution with Q2 degrees offreedom

I Assume X1 and X2 are independent

I Then, F defined as

F =X1/Q1X2/Q2

has a F distribution with (Q1,Q2) df

F FQ1,Q2

Rathelot

Introductory Econometrics

Distributions Basics of mathematical stats Confidence intervals

Population and sample

I Population: set of individuals, the population is defined bysome criteria; it is assumed infinite in the statisticalframework: rv Y

I Sample: drawn in the population, the sample is of finite size:rv {Y1 . . .Yn}

I Data sample: realizations of the previous rv in one particulardataset: {y1 . . . yn}

Rathelot

Introductory Econometrics

Distributions Basics of mathematical stats Confidence intervals

Random sample

I In basic statistical applications, the sample is often assumedto have been randomly drawn from the population

I If Y is a rv defined on the population from the density f ,{Y1 . . .Yn} is a random sample if these rv are independent,identically distributed from f

I The random nature of {Y1 . . .Yn}

Rathelot

Introductory Econometrics

Distributions Basics of mathematical stats Confidence intervals

Parametric estimation

I Let Y be a rv representing the population with a pdf f (y , )

I The pdf of Y is assumed to be known except for theparameter

I +: only have to estimate in order to know the pdf

I : sometimes, the assumption about the functional form ofthe pdf might be wrong

Rathelot

Introductory Econometrics

Distributions Basics of mathematical stats Confidence intervals

Estimators

I Given a sample {Y1 . . .Yn}, an estimator of is a function ofthe rv {Y1 . . .Yn}, that aims at measuring

I For instance, a natural estimator of E (Y ) is:

Y =1

n

ni=1

Yi

I Note that the estimator is a function of rv and is therefore arv itself

I For actual data outcomes, the estimate will be y = 1/n

i yi

Rathelot

Introductory Econometrics

Distributions Basics of mathematical stats Confidence intervals

Unbiasedness

I As any rv, an estimator has an expectation

I If this expectation is equal to the true value of the estimator,then the estimator is unbiased

I For instance, Y is unbiased as E (Y ) = E (Y )

I The bias of an estimator of is equal to E ()

Rathelot

Introductory Econometrics

Distributions Basics of mathematical stats Confidence intervals

Unbiasedness

I As any rv, an estimator has an expectation

I If this expectation is equal to the true value of the estimator,then the estimator is unbiased

I For instance, Y is unbiased as E (Y ) = E (Y )

I The bias of an estimator of is equal to E ()

Rathelot

Introductory Econometrics

Distributions Basics of mathematical stats Confidence intervals

The estimator of the variance

I The usual estimator of the variance of Y is

S2 =1

n 1

ni=1

(Yi Y )2

I It can be shown that S2 is unbiased

I Why dividing by n 1 and not n? Because Y is also estimated

Rathelot

Introductory Econometrics

Distributions Basics of mathematical stats Confidence intervals

The variance of the estimator

I What is a good estimator? Certainly an unbiased one

I Is it enough? An example

I Another criterion could be precision

I Need to compute the variance of the estimator

I Ex: Compute the variance of Y

Rathelot

Introductory Econometrics

Distributions Basics of mathematical stats Confidence intervals

The variance of the estimator

I What is a good estimator? Certainly an unbiased one

I Is it enough? An example

I Another criterion could be precision

I Need to compute the variance of the estimator

I Ex: Compute the variance of Y

Rathelot

Introductory Econometrics

Distributions Basics of mathematical stats Confidence intervals

Efficiency

I If and are two estimators of , it is better to chooser themore precise one

I If Var() Var() then is efficient relative to I If you consider biased estimators, comparing variance is not

enough

I One criterion (among others) is the mean square error (MSE)

MSE () = E [( )2] = Var() + [Bias()]2

Rathelot

Introductory Econometrics

Distributions Basics of mathematical stats Confidence intervals

Efficiency

I If and are two estimators of , it is better to chooser themore precise one

I If Var() Var() then is efficient relative to I If you consider biased estimators, comparing variance is not

enough

I One criterion (among others) is the mean square error (MSE)

MSE () = E [( )2] = Var() + [Bias()]2

Rathelot

Introductory Econometrics

Distributions Basics of mathematical stats Confidence intervals

Consistency

I Another issue with an estimator: how does it evolve when thesample size gets larger?

I An estimator is said to be consistent if it gets closer to thetrue value when the sample size increases

I Formally, is a consistent estimator of if, for every > 0,

P(| | > ) 0 as n

I Is said that the probability limit of is : plim()