Download - Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.

Transcript
Page 1: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.

Chapter 12Significance Tests in Practice

AP StatisticsHamilton/Mann

Page 2: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.

Introduction• In Chapter 11, we made the unrealistic claim that

we knew the population standard deviation σ.• In this chapter, we will stop making this assumption.• Just like with confidence intervals, when we no

longer know the population standard deviation σ, we must use a t distribution to carry out a significance test.

• Remember the use and abuse of tests from 11.3.• Also remember that we could be making a Type I or

Type II error whenever we use a significance test.• Think before you calculate!

Page 3: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.

CHAPTER 12 SECTION 1Tests about a Population Mean

HW: 12.2, 12.3, 12.4, 12.5, 12.6, 12.7, 12.9, 12.10, 12.11

Page 4: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.

Guinness Brewery• William S. Gosset was involved in experiments and

statistics to understand data for the Guinness brewery in Dublin, Ireland. He was trying to determine the best varieties of barley and hops for brewing? He ran into the problem of not knowing the population standard deviation σ. He observed that replacing σ by s and calling the result roughly Normal wasn’t accurate enough. After much work, Gosset developed what we now call the t distribution. Guinness allowed Gosset to publish his discoveries, but not under his own name. He used the name “Student,” and, as a result, it is sometimes referred to as Student’s t distribution.

Page 5: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.

• The t statistic has the same interpretation as any standardized statistic: it says how far is from its mean μ in standard deviation units.

• We are now going to learn to find P-values for a significance test about μ using the t table.

Page 6: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.

Determining P-values• Suppose we carry out a significance test of

based on a sample of size 20 and obtain t = 1.81.

• Since there were 20 observations, we would have df = 19. So look along the row with df = 19. Our t statistic falls between 1.729 and 2.093 which correspond to upper-tail values of 0.05 and 0.025.

• We can conclude that our P-value is between 0.025 and 0.05 because we were performing a one-sided test and it would be the area to the right, hence it would be the upper-tail value.

Page 7: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.

Determining P-values• Suppose we carry out a significance test of

based on a sample of size 37 and obtain t = -3.17.

• Since there were 37 observations, we would have df = 36. Since there is no 36 in the table, we look along the row with df = 30. Our t statistic falls between 3.030 and 3.385 which correspond to upper-tail values of 0.0025 and 0.001.

• Since it is a two-sided test, we have to find the probability less than -3.17 or greater than 3.17. Due to symmetry, the lower tail value would be the same as the upper-tail value, so we just double both of them. So we can conclude that our P-value is between 0.002 and 0.005.

Page 8: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.

The One-Sample t-test• In significance tests as in confidence intervals, we

allow for unknown σ by using the standard error and replacing z by t.

Page 9: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.

Testing• Now we can do a realistic analysis of data produced

to test a claim about an unknown population mean.• Again, we need to follow the steps in our Inference

Toolbox.1. Hypotheses – What is the population of interest and

what are the hypotheses we are testing?2. Conditions – SRS, Normality and Independence3. Calculations – Find P-value4. Interpretation – Connection, Conclusion, Context

Page 10: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.

Sweet Cola• Diet colas use artificial sweeteners to avoid sugar.

These sweeteners gradually lose their sweetness over time. Manufacturers therefore test new colas for loss of sweetness before marketing them. Trained tasters (wouldn’t you love this job) sip the cola along with drinks of standard sweetness and score the cola on a “sweetness scale” of 1 to 10. The cola is then stored for a month at high temperature to imitate the effect of storing it at room temperature for four months. Each taster will then score the cola again after storage. Our data are the differences (score before storage minus score after storage) in the tasters’ scores. The bigger these differences, the bigger the loss of sweetness.

Page 11: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.

Sweet Cola• Here are the sweetness losses for a new cola, as

measured by 10 different sweetness tasters.

• Notice that most are positive, indicating a loss of sweetness.

• Is there good evidence that the cola lost sweetness in storage?

• Step 1: We are interested in the average difference in sweetness between the before storage and after storage sweetness score.

2.0 0.4 0.7 2.0 -0.4 2.2 -1.3 1.2 1.1 2.3

Page 12: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.

Sweet Cola• Step 2 – Conditions – Since we do not know the

standard deviation of sweetness loss in the population of tasters, we must use a one-sided t-test.– SRS – We must be willing to treat our 10 tasters as an

SRS from the population of tasters if we want to draw conclusions about tasters in general. The tasters all have the same training. So even though we don’t have an actual SRS, we are willing to act as if we did.

– Normality – The sample is too small to effectively check normality. The stemplot and a boxplot show left skewedness, but no gaps or outliers.

– Independence – It is reasonable that different tasters would have results independent from each other.

Page 13: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.

Sweet Cola• Step 3 – Calculations

• Since our t-value is 2.70 and we have 10 observations, our P-value would be between 0.01 and 0.02. Using the calculator, we can get the exact value of 0.0123.

• Step 4 – Interpretation – A P-value between 0.01 and 0.02 is quite small and gives good evidence against H0. Therefore, we reject H0, and conclude that the cola has lost sweetness during storage.

Page 14: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.

T-Procedures• Because t-procedures are so common, all statistical

software packages will do the calculations for you. • The next two slides have the calculations as

performed by 4 different statistical software packages:

1.DataDesk2.Fathom3.Minitab4.CrunchIt!

Page 15: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.
Page 16: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.
Page 17: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.

Two-Tailed Example• An investor with a stock portfolio worth several

hundred thousand dollars sued his broker because lack of diversification in his portfolio led to poor performance. The table below gives the rates of return for the 39 months the account was managed by the broker.

• An arbitration panel compared these returns with the average of the Standard & Poor’s 500 stock index for the same period. Consider the 39 monthly returns as a random sample from the monthly returns the broker would generate if he managed the account forever. Are these returns compatible with a population mean of

the S&P 500 average?

-8.36 1.63 -2.27 -2.93 -2.70 -2.93 -9.14 -2.64 6.82 -2.35 -3.58 6.13 7.00

-15.25 -8.66 -1.03 -9.16 -1.25 -1.22 -10.27 -5.11 -0.80 -1.44 1.28 -0.65 4.34

12.22 -7.21 -0.09 7.34 5.04 -7.24 -2.14 -1.01 -1.41 12.03 -2.56 4.33 2.35

Page 18: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.

Two-Tailed Example• Step 1 – Hypotheses

where μ is the mean return for all possible months that the broker could manage this account

• Step 2 – Conditions – Since we don’t know σ, we must use t-procedures.– SRS – We were told to assume it was a random sample.– Normality – Since we have 39 observations, the Central

Limit Theorem says that it is approximately Normal. A boxplot and a histogram verify no outliers and no strong skewness.

– Independence – This is a matter of judgment. Would these 39 months represent independent observations?

Page 19: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.

Two-Tailed Example• Step 3 – Calculations

• Since our t-value is -2.14 and we have 39 observations, we would get an upper-tail value between 0.02 and 0.025. Since it is a two-tailed test, our P-value would be between 0.04 and 0.05

• Step 4 – Interpretation – The mean monthly returns for this client’s account differs significantly from the S&P 500 for the same period (t = -2.14, P<0.05).

• Software outputs are on the next two slides!

Page 20: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.
Page 21: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.
Page 22: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.

Estimating Mean Stock Return – C.I.• The mean monthly return on the client’s portfolio was

and the standard deviation was Our resulting 95% confidence interval is

• Because the S&P 500 return, 0.95%, falls outside of our interval, we know that μ differs significantly from 0.95% at the α = 0.05 level. Since the S&P 500 showed a mean gain of 0.95% during this time period, we can say with 95% confidence that the underperformance of this portfolio is between 0.09% and 4.01% per month. This estimate helps to determine the compensation owed to the investor.

Page 23: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.
Page 24: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.
Page 25: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.

Paired t Tests• In the taste test Example, the same 10 tasters rated

before and after sweetness. Since the data were paired by taster, we performed a one-sample t test on the differences.

• That is, we used a paired t test.• We are now going to look at another example of a

paired t test.

Page 26: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.

Floral Scents and Learning• We hear that listening to Mozart improves students’

performance on tests. Perhaps pleasant odors have a similar effect. To test this idea, 21 subjects worked a paper-and-pencil maze while wearing a mask. The mask was either unscented or carried a floral scent. The response variable is their average time on three trials. Each subject worked the maze with both masks, in a random order. The randomization is important because subjects tend to improve their times as they work a maze repeatedly. The table on the next slide gives the subjects’ average times with both masks.

Page 27: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.

Floral Scents and LearningSubject Unscented Scented Difference Subject Unscented Scented Difference

1 30.60 37.97 -7.37 12 58.93 83.50 -24.57

2 48.43 51.57 -3.14 13 54.47 38.30 16.17

3 60.77 56.67 4.10 14 43.53 51.37 -7.84

4 36.07 40.47 -4.40 15 37.93 29.33 8.60

5 68.47 49.00 19.47 16 43.50 54.27 -10.77

6 32.43 43.23 -10.80 17 87.70 62.73 24.97

7 43.70 44.57 -0.87 18 53.53 58.00 -4.47

8 37.10 28.40 8.70 19 64.30 52.40 11.90

9 31.17 28.23 2.94 20 47.37 53.63 -6.26

10 51.23 68.47 -17.24 21 53.67 47.00 6.67

11 65.40 51.10 14.30

• To analyze these data, subtract the scented times from the unscented times. Therefore a positive value indicates that the subject did better wearing the scented mask.

Page 28: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.

Floral Scenting and Learning• Step 1: Hypotheses – μ is the mean difference in the

population from which the subjects were drawn.

• Step 2: Conditions – Since we don’t know σ, we must use t procedures.– SRS – The data come from a randomized matched pairs

design, which means we can attribute any difference to the treatment. We can only generalize the results to the population if the sample is an SRS.

– Normality – Since it is not large enough, we must look at a stemplot and histogram to see if it is reasonably Normal. It has no outliers or gaps and appears Normal.

– Independence – It seems reasonable that one subjects difference in average mean completion time with the two different masks would be independent of another subjects.

Page 29: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.

Floral Scenting and Learning• Step 3 – Calculations

– For this t value, our P-value would be greater than 0.25.

• Step 4 – Interpretation– Since this P-value is large, we fail to reject H0. Therefore,

there is not enough evidence to conclude that scented masks improve performance.

• The next three slides contain statistical software printouts for the significance test.

Page 30: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.
Page 31: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.
Page 32: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.
Page 33: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.

One Sample t Test: Robustness and Power• Recall from Section 10.2 that t procedures are

robust against non-Normality of the population except when outliers or strong skewness are present.

• As the sample size increases, the Central Limit Theorem ensures that the distribution of the sample mean becomes more nearly Normal and that the t distribution becomes more accurate for calculating P-values.

• Review the guidelines in the box “Using the t Procedures” on p. 655.

Page 34: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.

One Sample t Test: Robustness and Power• The power of a statistical test measures its ability to

detect deviations from the null hypothesis.• In practice, we carry out the test in the hope of

showing that the null hypothesis is false, so higher power is important.

• The power of the one-sample t test against a specific alternative value of the population mean μ is the probability that the test will reject the null hypothesis when the mean has this alternative value.

Page 35: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.

CHAPTER 12 SECTION 2Tests about a Population Proportion

HW: 12.23, 12.24, 12.26, 12.29, 12.30, 12.31, 12.32

Page 36: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.

Tests about a Population Proportion• When the three important conditions are met, the

sampling distribution of is approximately Normal with mean and standard deviation

– SRS– Normality – np and n(1-p) both greater than 10– Independence

• For confidence intervals since we were trying to estimate p, we replaced p with in the standard deviation formula which gave us the standard error.

Page 37: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.

Tests about a Population Proportion• Now, we are performing a significance test. In a

significance test, our null hypothesis specifies a value for p which we call p0.

• So we will use p0 to find the standard deviation since we are assuming that it is correct.

• This means that our statistic is

Page 38: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.

Significance Test for a Proportion

Page 39: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.

Work Stress• According to the National Institute for Occupational

Safety and Health, job stress poses a major threat to the health of workers. A national survey of restaurant employees found that 75% said that work stress had a negative impact on their personal lives. A random sample of 100 employees from a large chain finds that 68 answer “Yes” when asked, “Does work stress have a negative impact on you personal life?” Is this good reason to think that the proportion of all employees in this chain who would say “Yes” differs from the national proportion of 0.75?

Page 40: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.

Work Stress• Step 1 – Hypotheses

• Step 2 – We should use a one-proportion z-test.– SRS – We are told it was an SRS.– Normality – The expected number of “Yes” and “No”

responses are 75 and 25 which are both larger than 10.– Independence – This large chain must have at least 1000

employees.

Page 41: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.

Work Stress• Step 3 – Calculations

• Since we are testing that it is not equal to, we must find the probability that it is less than -1.62 or greater than 1.62.

• Step 4 – Interpretation– Since our P-value of 0.1052 is fairly large, we would fail

to reject the null hypothesis. Therefore, there is no reason to believe that the proportion of workers at the large restaurant chain who suffer from work stress is different than the national survey result of 0.75.

Page 42: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.

Work Stress

Page 43: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.

Work Stress

Page 44: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.

Work Stress• For the work stress example, we arbitrarily chose a

response of “Yes” to be a success and “No” to be a failure.

• What would happen if we reversed these.• Let’s repeat the significance test with “No” being a

success. The national comparison value for the significance test will now be 0.25, the proportion in the national sample who said “No.”

Page 45: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.

Work Stress, Again• Step 1 – Hypotheses

• Step 2 – We should use a one-proportion z-test.– SRS – We are told it was an SRS.– Normality – The expected number of “No” and “Yes”

responses are 25 and 75 which are both larger than 10.– Independence – This large chain must have at least 1000

employees.

Page 46: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.

Work Stress, Again• Step 3 – Calculations

• Since we are testing that it is not equal to, we must find the probability that it is less than -1.62 or greater than 1.62.

• Step 4 – Interpretation– Since our P-value of 0.1052 is fairly large, we would fail

to reject the null hypothesis. Therefore, there is no reason to believe that the proportion of workers at the large restaurant chain who suffer from work stress is different than the national survey result of 0.75.

Page 47: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.

Work Stress, Again• When we interchanged “Yes” and “No,” we simply

changed the sign of the test statistic z. Our P-value remained the same.

• These results are true in general. Our conclusion does not depend on an arbitrary choice of success and failure.

Page 48: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.

Significance Test Results• The results of a significance test will often have

limited use.• Obviously, we would never expect the experiences

of a sample to be exactly the same as the overall population.

• If our sample is sufficiently large, however, we will have sufficient power to detect a very small difference.

• On the other hand, if our sample is very small, we may be unable to detect differences that could be very important.

• This is why we prefer to include a confidence interval as part of our analysis.

Page 49: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.

Confidence Intervals Provide More Info• A confidence interval allows us to see what other

values of p are compatible with the sample results.• This is why we will calculate a confidence interval.

Page 50: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.

Estimating Work Stress• The restaurant worker survey found that 68 out of 100

employees agreed that work stress had a negative impact on their personal lives. So we want to create a 95% confidence interval.

• Step 1 – Population– We want to estimate the proportion of restaurant workers

who believe that work stress had a negative impact on their personal lives.

• Step 2 – We should use a one-proportion z-interval.– SRS – We are told it was an SRS.– Normality – We have 68 successes and 32 failures in our

sample. Therefore it is approximately Normal.– Independence – This large chain must have at least 1000

employees.

Page 51: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.

Estimating Work Stress• Note: Checking normality for a confidence interval

and a significance test are different. For the confidence interval we check that

while we have to check that is true for a significance test

since we have to assume that the null hypothesis is true.

Page 52: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.

Work Stress, Again• Step 3 – Calculations

• Step 4 – Interpretation– We are 95% confident that between 59% and 77% of the

restaurant chain’s employees feel that work related stress is damaging their personal lives.

Page 53: Chapter 12 Significance Tests in Practice AP Statistics Hamilton/Mann.

Estimating Work Stress• The confidence interval gives us much more

information than the significance test.• The confidence interval tells us which values of p are

consistent with the sample results.• Notice that we use the standard error to create a

confidence interval while we use the hypothesized value to calculate the z statistic.

• As a result, we do not have the nice relationship between a confidence interval and a two-tailed significance test like we did for means. The results are still very close, but are not as exact as they were for means.

• Our confidence interval (0.59, 0.77) gives an approximate range of p0’s that would not be rejected by a test at the α = 0.05 significance level.