Transcript

Basics of Significance Testing

4TESTS OF SIGNIFICANCE: THE BASICS

SIGNIFICANCE TESTING

• Also called “hypothesis testing”• Objective: to test a claim about parameter μ

(population mean)• Procedure:

A.State hypotheses H0 and Ha

B.Calculate test statisticC.Convert test statistic to P-value and interpretD.Consider significance level (optional)

Basics of Significance Testing 2

HYPOTHESES• H0 (null hypothesis) claims “no difference”

• Ha (alternative hypothesis) contradicts the null

• Example: We test whether a population gained weight on average…

H0: no average weight gain in populationHa: H0 is wrong (i.e., “weight gain”)

• Next collect data quantify the extent to which the data provides evidence against H0

Basics of Hypothesis Testing 3

ONE-SAMPLE TEST OF MEAN• To test a single mean, the null hypothesis is

H0: μ = μ0, where μ0 represents the “null value” (null value comes from the research question, not from data!)

• The alternative hypothesis can take these forms: Ha: μ > μ0 (one-sided to right) orHa: μ < μ0 (one-side to left) or Ha: μ ≠ μ0 (two-sided)

• For the weight gain illustrative example:H0: μ = 0 Ha: μ > 0 (one-sided) or Ha: μ ≠ μ0 (two-sided)Note: μ0 = 0 in this example

Basics of Significance Testing 4

P-VALUE • The P value or calculated probability is the estimated

probability of rejecting the null hypothesis (H0) of a study question when that hypothesis is true. The p-value is a number between 0 and 1

• Smaller-and-smaller P-values → stronger-and-stronger evidence against H0

• Conventions for interpretation• Small p-value (typically ≤ 0.05) indicates strong evidence

against the null hypothesis, so you reject the null hypothesis.

• A large p-value (> 0.05) indicates weak evidence against the null hypothesis, so you fail to reject the null hypothesis.

• p-values very close to the cutoff (0.05) are considered to be marginal (could go either way).

Basics of Significance Testing 5

SIGNIFICANCE LEVEL• α ≡ threshold for “significance”• We set α• For example, if we choose α = 0.05, we

require evidence so strong that it would occur no more than 5% of the time when H0 is true

• Decision ruleP ≤ α statistically significant evidenceP > α nonsignificant evidence

• For example, if we set α = 0.01, a P-value of 0.0006 is considered significant

Basics of Significance Testing 6

04/07/23 Basics of Significance Testing 7

QUESTIONS?