Introductory Econometrics - Session 3 - Distribution .DistributionsBasics of mathematical statsCon

download Introductory Econometrics - Session 3 - Distribution .DistributionsBasics of mathematical statsCon

of 31

  • date post

    30-Jul-2018
  • Category

    Documents

  • view

    218
  • download

    0

Embed Size (px)

Transcript of Introductory Econometrics - Session 3 - Distribution .DistributionsBasics of mathematical statsCon

  • Distributions Basics of mathematical stats Confidence intervals

    Introductory EconometricsSession 3 - Distribution and confidence intervals

    Roland Rathelot

    Sciences Po

    July 2011

    Rathelot

    Introductory Econometrics

  • Distributions Basics of mathematical stats Confidence intervals

    The Normal distribution

    I If X is a r.v. normally distributed, X N(, 2),

    fX (x) =1

    2exp

    [(x )

    2

    22

    ]I The pdf of the standard normal distribution is:

    (z) =12

    exp[z2/2

    ]

    Rathelot

    Introductory Econometrics

  • Distributions Basics of mathematical stats Confidence intervals

    Some properties of the Normal distribution

    I If X N(, 2) then (X )/ N(0, 1)I If X N(, 2) then aX + b N(a+ b, a22)I Any linear combination of independent, identically distributed

    normal rv has a normal distribution

    I X and Y are jointly normally distributed. In this case, if Xand Y are uncorrelated then they are independent

    Rathelot

    Introductory Econometrics

  • Distributions Basics of mathematical stats Confidence intervals

    Some properties of the Normal distribution

    I If X N(, 2) then (X )/ N(0, 1)I If X N(, 2) then aX + b N(a+ b, a22)I Any linear combination of independent, identically distributed

    normal rv has a normal distribution

    I X and Y are jointly normally distributed. In this case, if Xand Y are uncorrelated then they are independent

    Rathelot

    Introductory Econometrics

  • Distributions Basics of mathematical stats Confidence intervals

    Some properties of the Normal distribution

    I If X N(, 2) then (X )/ N(0, 1)I If X N(, 2) then aX + b N(a+ b, a22)I Any linear combination of independent, identically distributed

    normal rv has a normal distribution

    I X and Y are jointly normally distributed. In this case, if Xand Y are uncorrelated then they are independent

    Rathelot

    Introductory Econometrics

  • Distributions Basics of mathematical stats Confidence intervals

    Some properties of the Normal distribution

    I If X N(, 2) then (X )/ N(0, 1)I If X N(, 2) then aX + b N(a+ b, a22)I Any linear combination of independent, identically distributed

    normal rv has a normal distribution

    I X and Y are jointly normally distributed. In this case, if Xand Y are uncorrelated then they are independent

    Rathelot

    Introductory Econometrics

  • Distributions Basics of mathematical stats Confidence intervals

    The chi-square distribution

    I Let Zi , i = 1 . . .Q be Q independent rv distributed asstandard normal

    I Then, the sum of the squares of the Zi , X , is known to have achi-square distribution with Q degrees of freedom (df)

    X =Qi=1

    Z 2i 2q

    I Expectation and variance

    E[X ] = Q;V [X ] = 2Q

    Rathelot

    Introductory Econometrics

  • Distributions Basics of mathematical stats Confidence intervals

    The chi-square distribution

    I Let Zi , i = 1 . . .Q be Q independent rv distributed asstandard normal

    I Then, the sum of the squares of the Zi , X , is known to have achi-square distribution with Q degrees of freedom (df)

    X =Qi=1

    Z 2i 2q

    I Expectation and variance

    E[X ] = Q;V [X ] = 2Q

    Rathelot

    Introductory Econometrics

  • Distributions Basics of mathematical stats Confidence intervals

    The t distribution

    I Let Z have a standard normal distribution

    I Let X have a chi-square distribution with Q degrees offreedom

    I Assume Z and X are independent

    I Then, T defined as

    T =ZX/Q

    has a t distribution with Q df

    T tQ

    Rathelot

    Introductory Econometrics

  • Distributions Basics of mathematical stats Confidence intervals

    The t and the Normal

    I An approximate statement: The t is just a normal withthicker tails

    I The pdf of the t and the normal look close: just the thicknessof the tails differ

    I The lower the df of the t, the thicker the tails

    I When the df tends to infinity, the t tends to a normal

    Rathelot

    Introductory Econometrics

  • Distributions Basics of mathematical stats Confidence intervals

    The F distribution

    I Let X1 have a chi-square distribution with Q1 degrees offreedom

    I Let X2 have a chi-square distribution with Q2 degrees offreedom

    I Assume X1 and X2 are independent

    I Then, F defined as

    F =X1/Q1X2/Q2

    has a F distribution with (Q1,Q2) df

    F FQ1,Q2

    Rathelot

    Introductory Econometrics

  • Distributions Basics of mathematical stats Confidence intervals

    Population and sample

    I Population: set of individuals, the population is defined bysome criteria; it is assumed infinite in the statisticalframework: rv Y

    I Sample: drawn in the population, the sample is of finite size:rv {Y1 . . .Yn}

    I Data sample: realizations of the previous rv in one particulardataset: {y1 . . . yn}

    Rathelot

    Introductory Econometrics

  • Distributions Basics of mathematical stats Confidence intervals

    Random sample

    I In basic statistical applications, the sample is often assumedto have been randomly drawn from the population

    I If Y is a rv defined on the population from the density f ,{Y1 . . .Yn} is a random sample if these rv are independent,identically distributed from f

    I The random nature of {Y1 . . .Yn}

    Rathelot

    Introductory Econometrics

  • Distributions Basics of mathematical stats Confidence intervals

    Parametric estimation

    I Let Y be a rv representing the population with a pdf f (y , )

    I The pdf of Y is assumed to be known except for theparameter

    I +: only have to estimate in order to know the pdf

    I : sometimes, the assumption about the functional form ofthe pdf might be wrong

    Rathelot

    Introductory Econometrics

  • Distributions Basics of mathematical stats Confidence intervals

    Estimators

    I Given a sample {Y1 . . .Yn}, an estimator of is a function ofthe rv {Y1 . . .Yn}, that aims at measuring

    I For instance, a natural estimator of E (Y ) is:

    Y =1

    n

    ni=1

    Yi

    I Note that the estimator is a function of rv and is therefore arv itself

    I For actual data outcomes, the estimate will be y = 1/n

    i yi

    Rathelot

    Introductory Econometrics

  • Distributions Basics of mathematical stats Confidence intervals

    Unbiasedness

    I As any rv, an estimator has an expectation

    I If this expectation is equal to the true value of the estimator,then the estimator is unbiased

    I For instance, Y is unbiased as E (Y ) = E (Y )

    I The bias of an estimator of is equal to E ()

    Rathelot

    Introductory Econometrics

  • Distributions Basics of mathematical stats Confidence intervals

    Unbiasedness

    I As any rv, an estimator has an expectation

    I If this expectation is equal to the true value of the estimator,then the estimator is unbiased

    I For instance, Y is unbiased as E (Y ) = E (Y )

    I The bias of an estimator of is equal to E ()

    Rathelot

    Introductory Econometrics

  • Distributions Basics of mathematical stats Confidence intervals

    The estimator of the variance

    I The usual estimator of the variance of Y is

    S2 =1

    n 1

    ni=1

    (Yi Y )2

    I It can be shown that S2 is unbiased

    I Why dividing by n 1 and not n? Because Y is also estimated

    Rathelot

    Introductory Econometrics

  • Distributions Basics of mathematical stats Confidence intervals

    The variance of the estimator

    I What is a good estimator? Certainly an unbiased one

    I Is it enough? An example

    I Another criterion could be precision

    I Need to compute the variance of the estimator

    I Ex: Compute the variance of Y

    Rathelot

    Introductory Econometrics

  • Distributions Basics of mathematical stats Confidence intervals

    The variance of the estimator

    I What is a good estimator? Certainly an unbiased one

    I Is it enough? An example

    I Another criterion could be precision

    I Need to compute the variance of the estimator

    I Ex: Compute the variance of Y

    Rathelot

    Introductory Econometrics

  • Distributions Basics of mathematical stats Confidence intervals

    Efficiency

    I If and are two estimators of , it is better to chooser themore precise one

    I If Var() Var() then is efficient relative to I If you consider biased estimators, comparing variance is not

    enough

    I One criterion (among others) is the mean square error (MSE)

    MSE () = E [( )2] = Var() + [Bias()]2

    Rathelot

    Introductory Econometrics

  • Distributions Basics of mathematical stats Confidence intervals

    Efficiency

    I If and are two estimators of , it is better to chooser themore precise one

    I If Var() Var() then is efficient relative to I If you consider biased estimators, comparing variance is not

    enough

    I One criterion (among others) is the mean square error (MSE)

    MSE () = E [( )2] = Var() + [Bias()]2

    Rathelot

    Introductory Econometrics

  • Distributions Basics of mathematical stats Confidence intervals

    Consistency

    I Another issue with an estimator: how does it evolve when thesample size gets larger?

    I An estimator is said to be consistent if it gets closer to thetrue value when the sample size increases

    I Formally, is a consistent estimator of if, for every > 0,

    P(| | > ) 0 as n

    I Is said that the probability limit of is : plim()