HS 167 Basics of Hypothesis Testing 1

Basics of Hypothesis Testing(a) Review of Inferential Basics

(b) Hypothesis Testing Procedure

(c) One-Sample z Test (σ known)

(d) One-sample t test (σ estimated with s)

HS 167 Basics of Hypothesis Testing 2

Review of Some Basics

Population all possible valuesSample a portion of the population Statistical inference generalizing from a sample to a population with calculated degree of certainty Two forms of statistical inference

Estimation Hypothesis testing

Parameter a numerical characteristic of a population, e.g., population mean µ, population standard deviation σStatistic a numerical characteristic calculated in the sample, e.g., sample mean x-bar, sample standard deviation s

HS 167 Basics of Hypothesis Testing 3

Recall the distinction between parameters and statistics

Parameters Statistics

Source Population Sample

Notation Greek ()

Roman (xbar, s)

Vary No Yes

Calculated No Yes

HS 167 Basics of Hypothesis Testing 4

Recall how sample means (xbars) vary

The sampling distribution of the mean (SDM) tends to be Normal with mean µ and a standard deviation called the SE

),(~ SENxn

SE

HS 167 Basics of Hypothesis Testing 5

Hypothesis Testing Procedure

A. HypothesesB. Test StatisticC. P-valueD. Significance level (optional)

HS 167 Basics of Hypothesis Testing 6

Step A. Hypotheses

Take the research question and convert it to statistical hypotheses State statistical hypotheses in null and alternative forms H0 Null hypothesis “no difference in the

population” Ha Alternative hypothesis “difference in

the population”

Seek evidence against H0 as a way of bolstering H1

HS 167 Basics of Hypothesis Testing 7

Step B. Test StatisticCalculate the appropriate test statistic There are many types of test statistics, depending on the conditions of the dataThis Chapter introduces this particular one-sample z statistic:

nSE

SE

x

and

trueis hypothesis null theassumingmean population the where

z

0

0stat

HS 167 Basics of Hypothesis Testing 8

Step C. P-valueConvert zstat to P-value using table or softwareThe P-value is the AUC in the tail of the SDM beyond the test statistics.The P-value answer the question: What is the probability of the observed test statistics or one that is more extreme assuming Hassuming H00 is true is true?Small P-value good evidence against H0 Although it unwise to draw too-firm Although it unwise to draw too-firm cutoffs, here are guidelines for the cutoffs, here are guidelines for the beginner:beginner:

P > 0.10 evidence against H0 not significant

0.05 < P 0.10 evidence against H0 marginally significant

0.01 < P 0.05 evidence against H0 significant

P 0.01 evidence against H0 highly significant

Smaller and smaller P-values provide stronger and stronger evidence against

HS 167 Basics of Hypothesis Testing 9

Step D. Significance level (optional)

Declare acceptable false rejection (Type I) error rate α

α can be set at any level (e.g., 1 in 20, 1 in 50, 1 in 100)

When P < α reject H0 at α level of significance

HS 167 Basics of Hypothesis Testing 10

Illustrative Example: Lake Wobegon, Study Design

Research question: Does a particular population of children have higher than average intelligence scores?Study design

Weschler intelligence scores are Normal with µ = 100 and = 15

Take a SRS Measure WIS {116, 128, 125, 119, 89, 99, 105,

116, 118} Calculate sample mean: x-bar = 112.8 Ask: Does this provide sufficient evidence that

population mean (μ) is greater than expected (100)?

HS 167 Basics of Hypothesis Testing 11

Illustrative Example: Lake Wobegon, Step A

Under the null hypothesis of “no difference” H0: µ = 100

Under the alternative hypothesis of “difference” Ha: µ > 100

Note: (a) Hypotheses address the parameter (not the statistic) (b) The value of the parameter under H0 is based on the research question (NOT the data)

HS 167 Basics of Hypothesis Testing 12

Illustrative Example: Lake Wobegon, Step B

The test statistics for this problem is the one-sample z statisticThis statistic assumes H0 is correctIt then standardizes the sample mean (i.e., turns it into a z-score):

56.25

1008.112

59

15

0stat

SEM

xz

nSEM

HS 167 Basics of Hypothesis Testing 13

Illustrative Example: Lake Wobegon, Step C

The test statistic is converted to a P-valueAssuming H0 true, where does zstat fall on the curve?P value area under curve beyond zstat

Use Z table (or software) to find Pr(Z ≥ zstat) = Pr(Z ≥ 2.56) = 0.0052The P-value of 0.0052 provides strong (“highly significant”) evidence against H0

)1,0(~

)(~

under that Recall

stat

,0

0

Nz

SENx

H

HS 167 Basics of Hypothesis Testing 14

Illustrative Example: Lake Wobegon Step D (Optional)

P = 0.0052 is highly significant evidence against H0

The smaller the P, the more significant the evidence.

See guidelines in earlier slide.

HS 167 Basics of Hypothesis Testing 15

The one-sided alternative

The prior test made a supposition about the direction of the difference The test had a “one-sided H1”

We looked only at one side of the

SDM

HS 167 Basics of Hypothesis Testing 16

The two-sided alternative

Allows for unanticipated findings either up or down from expectedThe requires a two-two-sided test sided test The two-sided test looks at both tails Just double the one-sided PLake Wobegon example two-sided P = 2 × 0.0052 = 0.0104

HS 167 Basics of Hypothesis Testing 17

Illustrative example: “Anemia”

Research question: A public health official suspects a particular population is anemic Hemoglobin is a protein in red blood cells that

carries oxygen People with less than 12 g/dl hemoglobin are

anemic We know that hemoglobin levels in children are

Normal with standard deviation σ = 1.6 g/dl

Data: The researcher measures hemoglobin in 50 children and finds x-bar = 11.3

HS 167 Basics of Hypothesis Testing 18

Step A: Anemia example

We seek evidence against µ = 12. Thus, H0: µ = 12

The one-sided alternative is H1: µ < 12The two-sided alternative is H1: µ 12Let’s do a two-sided test (much more common in practice)

HS 167 Basics of Hypothesis Testing 19

Step B: Test statistic

10.3226.0

123.11

226.050

6.1

0stat

SEM

xz

nSEM

HS 167 Basics of Hypothesis Testing 20

Step C. P-value

Pr(Z < -3.10) = 0.0010Double this b/c the alternative is two-sided: P = 2 × 0.0010 = 0.0020Comment: Under H0, xbar~N(12, 0.226), I’ll draw this on the board so you see that an x-bar of 11.3 is in the far left tail

HS 167 Basics of Hypothesis Testing 21

Step D: Conclusion

P = 0.0020

evidence against H0 is highly significant

HS 167 Basics of Hypothesis Testing 22

Conditions necessary for z test

Simple random sample (SRS)Normal population or large sample known (not calculated)

What do you do when σ is not known? Use a t procedure!

HS 167 Basics of Hypothesis Testing 23

Diabetic weight illustrative example

Claim: “diabetics are over-weight”Data are “% of ideal body weight” n = 18 Sample mean (x-bar) = 112.778 Sample standard deviation (s) =

14.424

HS 167 Basics of Hypothesis Testing 24

Step A: Hypotheses (diabetic weight)

Research claim is “diabetics are over-weight”Convert claim to a null hypothesis “Diabetics are not overweight” Not overweight = 100 of ideal body weight Therefore, H0: µ = 100

Alternative hypothesis can be H1: µ 100 (two-sided) H1: µ > 100 (one-sided to right) H1: µ < 100 (one-sided to left)

HS 167 Basics of Hypothesis Testing 25

Step B: Test statistic

ndf n

ssem

sem

xt

1

,hypothesis null under themean population where

0

0stat

tstat tells you how many standard errors the sample mean falls from hypothesized population mean

171181

76.3400.3

100778.112

400.318

424.14

0stat

ndfsem

xt

n

ssem

HS 167 Basics of Hypothesis Testing 26

Step C: P value (Diabetic weight)

t table: wedge tstat between t landmarksExample: tstat = 3.76 on t with 17 df falls between 3.646 (right tail 0.001) and 3.965 (right tail 0.0005) Two-tailed P is twice the one-tailed values: P less than 0.002 and more than 0.001P = 0.0016 (via software)

HS 167 Basics of Hypothesis Testing 27

Step D: Conclusion

P = 0.0016 provides highly significant evidence against H0

HS 167 Basics of Hypothesis Testing 28

Consequences of test decisions

TRUTHDECISION

H0 true H0 false

Retain H0 Correct retention

type II error

Reject H0type I error

Correct rejection

Probability (type I error) =

Probability (type II error) =

HS 167 Basics of Hypothesis Testing 29

Type II errors (β)

We have considered only type I () errors In the next chapter we will take up the issue of type II () errors Pr(rejecting a false H0) The complement of is 1 –

”Power”

HS 167 Basics of Hypothesis Testing 30

Fallacies of hypothesis testing

Failure to reject H0 = acceptance of H0 (WRONG!)

P value = probability H0 is incorrect (WRONG!)Statistical significance implies biological or social importance (WRONG!)

HS 167 Basics of Hypothesis Testing 31

Beware!

Hypothesis testing addresses random error only. It does not account for many problems encountered in practice, such as measurement error and sampling biases