Statistical Inference

Part IV

Statistical Inference

As of Oct 2, 2019 Seppo Pynnönen Econometrics I

Statistical Inference

Sampling Distributions of the OLS Estimator

1 Statistical Inference

Sampling Distributions of the OLS Estimator

Testing for single population coefficients, the t-test

Testing Against One-Sided Alternatives

Two-Sided Alternatives

Other Hypotheses About βj

p-values

Economic, or Practical, versus Statistical Significance

Confidence Intervals for the Coefficients

Simultaneous Testing for Multiple Coefficients, The F -test

Testing General Linear Restrictions

On Reporting the Regression Results

Seppo Pynnönen Econometrics I

Statistical Inference

Sampling Distributions of the OLS Estimator

Regression model

yi = β0 + β1xi1 + · · ·+ βkxik + ui . (1)

To the assumptions 1–5 we add

Assumption 6: The error component u is independent of x1, . . . xk and

u ∼ N(0, σ2u). (2)

Seppo Pynnönen Econometrics I

Statistical Inference

Sampling Distributions of the OLS Estimator

Exploring further 4.1 (W 5e, p. 113)

Suppose that u is independent of the explanatory variables, and it takes on the values

−2, −1, 0, 1, and 2 with probability of 1/5. Does this violate assumptions 1–5 (Gauss-Markov assumptions)? Does this violate assumption 6?

Seppo Pynnönen Econometrics I

Statistical Inference

Sampling Distributions of the OLS Estimator

Remark 1

Assumption 6 implies E[u|x1, . . . , xk ] = E[u] = 0 and var[u|x1, . . . , xk ] = var[u] = σ2u, i.e Assumptions 1 and 2.

Remark 2

Assumption 3, i.e., cov[ui , uj ] = 0 together with Assumption 6 implies

that u1, . . . , un are independent.

Remark 3

Under assumptions 1–6 the OLS estimators β̂1, . . . , β̂k are Minimum

Variance Unbiased Estimators (MVUE). That is they are best among all

unbiased estimators (not only linear).

Seppo Pynnönen Econometrics I

Statistical Inference

Sampling Distributions of the OLS Estimator

Remark 4

y |x ∼ N(β0 + β1x1 + · · ·+ βkxk , σ2u), (3)

where x = (x1, . . . , xk)′ and y |x means ”conditional on x”.

Theorem 1

Under the assumptions 1–6, conditional on the sample values of the explanatory variables

β̂j ∼ N(βj , σ2β̂j ) (4)

and therefore β̂j − βj σβ̂j

∼ N(0, 1), (5)

where σ2 β̂j

= var [ β̂j

] and σβ̂j =

√ var

[ β̂j

] .

Seppo Pynnönen Econometrics I

Statistical Inference

Testing for single population coefficients, the t-test

1 Statistical Inference

Sampling Distributions of the OLS Estimator

Testing for single population coefficients, the t-test

Testing Against One-Sided Alternatives

Two-Sided Alternatives

Other Hypotheses About βj

p-values

Economic, or Practical, versus Statistical Significance

Confidence Intervals for the Coefficients

Simultaneous Testing for Multiple Coefficients, The F -test

Testing General Linear Restrictions

On Reporting the Regression Results

Seppo Pynnönen Econometrics I

Statistical Inference

Testing for single population coefficients, the t-test

Theorem 2

Under the assumptions 1–6

β̂j − βj sβ̂j

∼ tn−k−1, (6)

(the t-distribution with n − k − 1 degrees of freedom) where sβ̂j = se(β̂j) and k + 1 is the number of estimated regression coefficients.

Remark 5

The only difference between (5) and (6) is that in the latter the standard

deviation parameter σβ̂j is replaced by its estimator sβ̂j .

Seppo Pynnönen Econometrics I

Statistical Inference

Testing for single population coefficients, the t-test

In most applications the interest lies in testing the null hypothesis:

H0 : βj = 0. (7)

The t-test statistic is

tβ̂j = β̂j sβ̂j , (8)

which is t-distributed with n − k − 1 degrees of freedom if the null hypothesis is true.

These ”t-ratios” are printed in standard computer output in regression applications.

Seppo Pynnönen Econometrics I

Statistical Inference

Testing for single population coefficients, the t-test

Example 1

Wage example computer output.

Dependent variable: log(wage)

Residuals:

Min 1Q Median 3Q Max

-2.21158 -0.36393 -0.07263 0.29712 1.52339

Coefficients:

Estimate Std. Error t value Pr(>|t|)

(Intercept) 0.583773 0.097336 5.998 3.74e-09 ***

educ 0.082744 0.007567 10.935 < 2e-16 ***

---

Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.4801 on 524 degrees of freedom

Multiple R-squared: 0.1858,Adjusted R-squared: 0.1843

F-statistic: 119.6 on 1 and 524 DF, p-value: < 2.2e-16

Seppo Pynnönen Econometrics I

Statistical Inference

Testing for single population coefficients, the t-test

1 Statistical Inference

Sampling Distributions of the OLS Estimator

Testing for single population coefficients, the t-test

Testing Against One-Sided Alternatives

Two-Sided Alternatives

Other Hypotheses About βj

p-values

Economic, or Practical, versus Statistical Significance

Confidence Intervals for the Coefficients

Simultaneous Testing for Multiple Coefficients, The F -test

Testing General Linear Restrictions

On Reporting the Regression Results

Seppo Pynnönen Econometrics I

Statistical Inference

Testing for single population coefficients, the t-test

One-sided alternatives H1 : βj > 0 (9)

or H1 : βj < 0. (10)

In the former the rejection rule is to reject the null hypothesis at the chosen significance level, α

tβ̂j > cα, (11)

where cα is the 1− α fractile (or percentile) from the t-distribution with n − k − 1 degrees of freedom, such that P(tβ̂j > cα|H0 is true) = α. α is called the significance level of the test. Typically α is 0.05 or 0.01, i.e., 5% or 1%.

In the case of (10) the H0 is rejected if

tβ̂j < −cα. (12)

These tests are one-tailed test.

Seppo Pynnönen Econometrics I

Statistical Inference

Testing for single population coefficients, the t-test

Exploring Further 4.2 (W 5ed, p. 118) (adapted)

Community loan approval rates, apprate, are modeled as

apprate = β0 + β1percmin + β3avginc + β4avgwlth + β5avgdebt + u,

where percmin is the percentage minority in the community, avginc is average income, avgwlth is average wealth, and avgdept is average dept obligations.

Suppose we are interested to investigate whether there is difference in loan approval rates across neighborhoods due to racial and ethnic composition, when average income, average wealth, and average dept have been controlled for. We are particularly interested in whether there is discrimination against minorities in loan approval rates.

How would you state the null hypothesis in terms of the above model? How do you state the alternative hypothesis?

Seppo Pynnönen Econometrics I

Statistical Inference

Testing for single population coefficients, the t-test

Example 2

In the wage example, test

H0 : βexper = 0 against H1 : βexper > 0.

Dependent variable: log(wage)

Coefficients:

Estimate Std. Error t value Pr(>|t|)

(Intercept) 0.284360 0.104190 2.729 0.00656 **

educ 0.092029 0.007330 12.555 < 2e-16 ***

exper 0.004121 0.001723 2.391 0.01714 *

tenure 0.022067 0.003094 7.133 3.29e-12 ***

---

Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.4409 on 522 degrees of freedom

Multiple R-squared: 0.316,Adjusted R-squared: 0.3121

F-statistic: 80.39 on 3 and 522 DF, p-value: < 2.2e-16

Seppo Pynnönen Econometrics I

Statistical Inference

Testing for single population coefficients, the t-test

Example 2 continues

Thus, β̂exper = 0.004121, sβ̂exper = 0.001723 and

tβ̂exper = β̂exper sβ̂exper

= 0.004121

0.001723 ≈ 2.39,

and because 2.39 > 2.33 = c.01 (from t-distribution with 522 df), we

reject the null hypothesis and infer that experience affects positively on

wages.

Seppo Pynnönen Econometrics I

Statistical Inference

Testing for single population coefficients, the t-test

1 Statistical Inference

Sampling Distributions of the OLS Estimator

Testing for single population coefficients, the t-test

Testing Against One-Sided Alternatives

Two-Sided Alternatives

Other Hypotheses About βj

p-values

Economic, or Practical, versus Statistical Significance

Confidence Intervals for the Coefficients

Simultaneous Testing for Multiple Coefficients, The F -test

Testing General Linear Restrictions

On Reporting t