Introductory Econometrics - Session 3 - Distribution · PDF fileDistributionsBasics of...
date post
30-Jul-2018Category
Documents
view
221download
0
Embed Size (px)
Transcript of Introductory Econometrics - Session 3 - Distribution · PDF fileDistributionsBasics of...
Distributions Basics of mathematical stats Confidence intervals
Introductory EconometricsSession 3 - Distribution and confidence intervals
Roland Rathelot
Sciences Po
July 2011
Rathelot
Introductory Econometrics
Distributions Basics of mathematical stats Confidence intervals
The Normal distribution
I If X is a r.v. normally distributed, X N(, 2),
fX (x) =1
2exp
[(x )
2
22
]I The pdf of the standard normal distribution is:
(z) =12
exp[z2/2
]
Rathelot
Introductory Econometrics
Distributions Basics of mathematical stats Confidence intervals
Some properties of the Normal distribution
I If X N(, 2) then (X )/ N(0, 1)I If X N(, 2) then aX + b N(a+ b, a22)I Any linear combination of independent, identically distributed
normal rv has a normal distribution
I X and Y are jointly normally distributed. In this case, if Xand Y are uncorrelated then they are independent
Rathelot
Introductory Econometrics
Distributions Basics of mathematical stats Confidence intervals
Some properties of the Normal distribution
I If X N(, 2) then (X )/ N(0, 1)I If X N(, 2) then aX + b N(a+ b, a22)I Any linear combination of independent, identically distributed
normal rv has a normal distribution
I X and Y are jointly normally distributed. In this case, if Xand Y are uncorrelated then they are independent
Rathelot
Introductory Econometrics
Distributions Basics of mathematical stats Confidence intervals
Some properties of the Normal distribution
I If X N(, 2) then (X )/ N(0, 1)I If X N(, 2) then aX + b N(a+ b, a22)I Any linear combination of independent, identically distributed
normal rv has a normal distribution
I X and Y are jointly normally distributed. In this case, if Xand Y are uncorrelated then they are independent
Rathelot
Introductory Econometrics
Distributions Basics of mathematical stats Confidence intervals
Some properties of the Normal distribution
I If X N(, 2) then (X )/ N(0, 1)I If X N(, 2) then aX + b N(a+ b, a22)I Any linear combination of independent, identically distributed
normal rv has a normal distribution
I X and Y are jointly normally distributed. In this case, if Xand Y are uncorrelated then they are independent
Rathelot
Introductory Econometrics
Distributions Basics of mathematical stats Confidence intervals
The chi-square distribution
I Let Zi , i = 1 . . .Q be Q independent rv distributed asstandard normal
I Then, the sum of the squares of the Zi , X , is known to have achi-square distribution with Q degrees of freedom (df)
X =Qi=1
Z 2i 2q
I Expectation and variance
E[X ] = Q;V [X ] = 2Q
Rathelot
Introductory Econometrics
Distributions Basics of mathematical stats Confidence intervals
The chi-square distribution
I Let Zi , i = 1 . . .Q be Q independent rv distributed asstandard normal
I Then, the sum of the squares of the Zi , X , is known to have achi-square distribution with Q degrees of freedom (df)
X =Qi=1
Z 2i 2q
I Expectation and variance
E[X ] = Q;V [X ] = 2Q
Rathelot
Introductory Econometrics
Distributions Basics of mathematical stats Confidence intervals
The t distribution
I Let Z have a standard normal distribution
I Let X have a chi-square distribution with Q degrees offreedom
I Assume Z and X are independent
I Then, T defined as
T =ZX/Q
has a t distribution with Q df
T tQ
Rathelot
Introductory Econometrics
Distributions Basics of mathematical stats Confidence intervals
The t and the Normal
I An approximate statement: The t is just a normal withthicker tails
I The pdf of the t and the normal look close: just the thicknessof the tails differ
I The lower the df of the t, the thicker the tails
I When the df tends to infinity, the t tends to a normal
Rathelot
Introductory Econometrics
Distributions Basics of mathematical stats Confidence intervals
The F distribution
I Let X1 have a chi-square distribution with Q1 degrees offreedom
I Let X2 have a chi-square distribution with Q2 degrees offreedom
I Assume X1 and X2 are independent
I Then, F defined as
F =X1/Q1X2/Q2
has a F distribution with (Q1,Q2) df
F FQ1,Q2
Rathelot
Introductory Econometrics
Distributions Basics of mathematical stats Confidence intervals
Population and sample
I Population: set of individuals, the population is defined bysome criteria; it is assumed infinite in the statisticalframework: rv Y
I Sample: drawn in the population, the sample is of finite size:rv {Y1 . . .Yn}
I Data sample: realizations of the previous rv in one particulardataset: {y1 . . . yn}
Rathelot
Introductory Econometrics
Distributions Basics of mathematical stats Confidence intervals
Random sample
I In basic statistical applications, the sample is often assumedto have been randomly drawn from the population
I If Y is a rv defined on the population from the density f ,{Y1 . . .Yn} is a random sample if these rv are independent,identically distributed from f
I The random nature of {Y1 . . .Yn}
Rathelot
Introductory Econometrics
Distributions Basics of mathematical stats Confidence intervals
Parametric estimation
I Let Y be a rv representing the population with a pdf f (y , )
I The pdf of Y is assumed to be known except for theparameter
I +: only have to estimate in order to know the pdf
I : sometimes, the assumption about the functional form ofthe pdf might be wrong
Rathelot
Introductory Econometrics
Distributions Basics of mathematical stats Confidence intervals
Estimators
I Given a sample {Y1 . . .Yn}, an estimator of is a function ofthe rv {Y1 . . .Yn}, that aims at measuring
I For instance, a natural estimator of E (Y ) is:
Y =1
n
ni=1
Yi
I Note that the estimator is a function of rv and is therefore arv itself
I For actual data outcomes, the estimate will be y = 1/n
i yi
Rathelot
Introductory Econometrics
Distributions Basics of mathematical stats Confidence intervals
Unbiasedness
I As any rv, an estimator has an expectation
I If this expectation is equal to the true value of the estimator,then the estimator is unbiased
I For instance, Y is unbiased as E (Y ) = E (Y )
I The bias of an estimator of is equal to E ()
Rathelot
Introductory Econometrics
Distributions Basics of mathematical stats Confidence intervals
Unbiasedness
I As any rv, an estimator has an expectation
I If this expectation is equal to the true value of the estimator,then the estimator is unbiased
I For instance, Y is unbiased as E (Y ) = E (Y )
I The bias of an estimator of is equal to E ()
Rathelot
Introductory Econometrics
Distributions Basics of mathematical stats Confidence intervals
The estimator of the variance
I The usual estimator of the variance of Y is
S2 =1
n 1
ni=1
(Yi Y )2
I It can be shown that S2 is unbiased
I Why dividing by n 1 and not n? Because Y is also estimated
Rathelot
Introductory Econometrics
Distributions Basics of mathematical stats Confidence intervals
The variance of the estimator
I What is a good estimator? Certainly an unbiased one
I Is it enough? An example
I Another criterion could be precision
I Need to compute the variance of the estimator
I Ex: Compute the variance of Y
Rathelot
Introductory Econometrics
Distributions Basics of mathematical stats Confidence intervals
The variance of the estimator
I What is a good estimator? Certainly an unbiased one
I Is it enough? An example
I Another criterion could be precision
I Need to compute the variance of the estimator
I Ex: Compute the variance of Y
Rathelot
Introductory Econometrics
Distributions Basics of mathematical stats Confidence intervals
Efficiency
I If and are two estimators of , it is better to chooser themore precise one
I If Var() Var() then is efficient relative to I If you consider biased estimators, comparing variance is not
enough
I One criterion (among others) is the mean square error (MSE)
MSE () = E [( )2] = Var() + [Bias()]2
Rathelot
Introductory Econometrics
Distributions Basics of mathematical stats Confidence intervals
Efficiency
I If and are two estimators of , it is better to chooser themore precise one
I If Var() Var() then is efficient relative to I If you consider biased estimators, comparing variance is not
enough
I One criterion (among others) is the mean square error (MSE)
MSE () = E [( )2] = Var() + [Bias()]2
Rathelot
Introductory Econometrics
Distributions Basics of mathematical stats Confidence intervals
Consistency
I Another issue with an estimator: how does it evolve when thesample size gets larger?
I An estimator is said to be consistent if it gets closer to thetrue value when the sample size increases
I Formally, is a consistent estimator of if, for every > 0,
P(| | > ) 0 as n
I Is said that the probability limit of is : plim()