7/2/2015Basics of Significance Testing1 Chapter 15 Tests of Significance: The Basics.

18
06/11/22 Basics of Significance Testing 1 Chapter 15 Tests of Significance: The Basics

Transcript of 7/2/2015Basics of Significance Testing1 Chapter 15 Tests of Significance: The Basics.

Page 1: 7/2/2015Basics of Significance Testing1 Chapter 15 Tests of Significance: The Basics.

04/19/23 Basics of Significance Testing 1

Chapter 15

Tests of Significance: The Basics

Page 2: 7/2/2015Basics of Significance Testing1 Chapter 15 Tests of Significance: The Basics.

04/19/23 Basics of Significance Testing 2

Significance Testing

• Also called “hypothesis testing”

• Objective: to test a claim about parameter μ

• Procedure: A.State hypotheses H0 and Ha

B.Calculate test statistic

C.Convert test statistic to P-value and interpret

D.Consider significance level (optional)

Page 3: 7/2/2015Basics of Significance Testing1 Chapter 15 Tests of Significance: The Basics.

Basic Biostat 9: Basics of Hypothesis Testing 3

Hypotheses• H0 (null hypothesis) claims “no difference”

• Ha (alternative hypothesis) contradicts the null

• Example: We test whether a population gained weight on average…

H0: no average weight gain in populationHa: H0 is wrong (i.e., “weight gain”)

• Next collect data quantify the extent to which the data provides evidence against H0

Page 4: 7/2/2015Basics of Significance Testing1 Chapter 15 Tests of Significance: The Basics.

04/19/23 Basics of Significance Testing 4

One-Sample Test of Mean• To test a single mean, the null hypothesis is

H0: μ = μ0, where μ0 represents the “null value” (null value comes from the research question, not from data!)

• The alternative hypothesis can take these forms: Ha: μ > μ0 (one-sided to right) orHa: μ < μ0 (one-side to left) or Ha: μ ≠ μ0 (two-sided)

• For the weight gain illustrative example:H0: μ = 0 Ha: μ > 0 (one-sided) or Ha: μ ≠ μ0 (two-sided)

Note: μ0 = 0 in this example

Page 5: 7/2/2015Basics of Significance Testing1 Chapter 15 Tests of Significance: The Basics.

04/19/23 Basics of Significance Testing 5

Illustrative Example: Weight Gain• Let X ≡ weight gain • X ~N(μ, σ = 1), the

value of μ unknown

• Under H0, μ = 0

• Take SRS of n = 10

• σx-bar = 1 / √(10) = 0.316

• Thus, under H0 x-bar~N(0, 0.316)

Figure: Two possible xbars when H0 true

Page 6: 7/2/2015Basics of Significance Testing1 Chapter 15 Tests of Significance: The Basics.

04/19/23 Basics of Significance Testing 6

Take an SRS of size n from a Normal population. Population σ is known. Test H0: μ = μ0 with:

One-Sample z Statistic

x

μxz

μxz

00

ly Equivalent

For “weight gain” data, x-bar = 1.02, n = 10, and σ = 1

3.23

101

01.020

μxz

Page 7: 7/2/2015Basics of Significance Testing1 Chapter 15 Tests of Significance: The Basics.

04/19/23 Basics of Significance Testing 7

P-value • P-value ≡ the probability the test statistic would

take a value as extreme or more extreme than observed test statistic, when H0 is true

• Smaller-and-smaller P-values → stronger-and-stronger evidence against H0

• Conventions for interpretation

P > .10 evidence against H0 not significant.05 < P ≤ .10 evidence marginally significant

.01 < P ≤ .05 evidence against H0 significant

P ≤ .01 evidence against H0 very significant

Page 8: 7/2/2015Basics of Significance Testing1 Chapter 15 Tests of Significance: The Basics.

04/19/23 Basics of Significance Testing 8

P-Value

Convert z statistics to P-value :

• For Ha: μ> μ0

P = Pr(Z > zstat) = right-tail beyond zstat

• For Ha: μ< μ0 P = Pr(Z < zstat) = left tail beyond zstat

• For Ha: μμ0 P = 2 × one-tailed P-value

Page 9: 7/2/2015Basics of Significance Testing1 Chapter 15 Tests of Significance: The Basics.

04/19/23 Basics of Significance Testing 9

Illustrative Example

• z statistic = 3.23

• One-sided P = P(Z > 3.23) = 1−0.9994 = 0.0006

• Highly significant evidence against H0

Page 10: 7/2/2015Basics of Significance Testing1 Chapter 15 Tests of Significance: The Basics.

04/19/23 Basics of Significance Testing 10

• α ≡ threshold for “significance”• We set α• For example, if we choose α = 0.05, we require

evidence so strong that it would occur no more than 5% of the time when H0 is true

• Decision ruleP ≤ α statistically significant evidenceP > α nonsignificant evidence

• For example, if we set α = 0.01, a P-value of 0.0006 is considered significant

Significance Level

Page 11: 7/2/2015Basics of Significance Testing1 Chapter 15 Tests of Significance: The Basics.

04/19/23 Basics of Significance Testing 11

Summary

Page 12: 7/2/2015Basics of Significance Testing1 Chapter 15 Tests of Significance: The Basics.

04/19/23 Basics of Significance Testing 12

Illustrative Example: Two-sided test

1. Hypotheses: H0: μ = 0 against Ha: μ ≠ 0

2. Test Statistic:

3. P-value: P = 2 × Pr(Z > 3.23) = 2 × 0.0006 = 0.0012 Conclude highly significant evidence against H0

3.23

101

01.020

μxz

Page 13: 7/2/2015Basics of Significance Testing1 Chapter 15 Tests of Significance: The Basics.

04/19/23 Basics of Significance Testing 13

Relation Between Tests and CIs• For two-sided tests, significant results at the α-

level μ0 will fall outside (1–α)100% CI

• When α = .05 (1–α)100% = (1–.05)100% = 95% confidence

• When α = .01, (1–α)100% = (1–.01)100% = 99% confidence

• Recall that we tested H0: μ = 0 and found a two-sided P = 0.0012. Since this is significant at α = .05, we expect “0” to fall outside that 95% confidence interval … continued …

Page 14: 7/2/2015Basics of Significance Testing1 Chapter 15 Tests of Significance: The Basics.

04/19/23 Basics of Significance Testing 14

Relation Between Tests and CIs

1.64 to40.0

62.002.110

196.102.1

n

σzx

Recall: xbar = 1.02, n = 10, σ = 1. Therefore, a 95% CI for μ =

Since 0 falls outside this 95% CI the test of H0: μ = 0 is significant at α = .05

Page 15: 7/2/2015Basics of Significance Testing1 Chapter 15 Tests of Significance: The Basics.

04/19/23 Basics of Significance Testing 15

Example II: Job Satisfaction

The null hypothesis “no average difference” in the population of workers. The alternative hypothesis is “there is an average difference in scores” in the population.

H0: = 0 Ha: ≠ 0

This is a two-sided test because we are interested in differences in either direction.

Does the job satisfaction of assembly workers differ when their work is machine-paced rather than self-paced? A matched pairs study was performed on a sample of workers. Workers’ satisfaction was assessed in each setting. The response variable is the difference in satisfaction scores, self-paced minus machine-paced.

Page 16: 7/2/2015Basics of Significance Testing1 Chapter 15 Tests of Significance: The Basics.

04/19/23 Basics of Significance Testing 16

Illustrative Example II Job satisfaction scores follow a Normal

distribution with standard deviation = 60. Data from 18 workers gives a sample mean

difference score of 17. Test H0: µ = 0 against Ha: µ ≠ 0 with

1.20

1860

0170

μxz

Page 17: 7/2/2015Basics of Significance Testing1 Chapter 15 Tests of Significance: The Basics.

04/19/23 Basics of Significance Testing 17

Illustrative Example II

• Two-sided P-value = Pr(Z < -1.20 or Z > 1.20) = 2 × Pr (Z > 1.20)= (2)(0.1151) = 0.2302

• Conclude: 0.2302 chance we would see results this extreme when H0 is true evidence against H0 not strong (not significant)

Page 18: 7/2/2015Basics of Significance Testing1 Chapter 15 Tests of Significance: The Basics.

04/19/23 Basics of Significance Testing 18

Example II: Conf Interval Method

This 90% CI includes 0. Therefore, it is plausible that the true value of is 0 H0: µ = 0 cannot be rejected at α = 0.10.

40.26 to 6.26

23.261718

601.64517

n

σzx

Studying Job Satisfaction

A 90% CI for μ is