Hypothesis test flow chart

frequencydata

Measurement scale

number of variables

1

basic χ2 test(19.5)

Table I

χ2 test for independence(19.9) Table I

2correlation (r)number of correlations

1Test H0: r=0(17.2)

Table G

2

Test H0: r1= r2

(17.4) Tables H and A

number of means

Mea

ns

Do you know s?

1Yesz -test(13.1) Table A

t -test(13.14)Table D

2

independentsamples?

Yes

Test H0: m1= m2

(15.6) Table D

No

Test H0: D=0(16.4)

Table D

More than 2 number of factors

1

1-way ANOVACh 20

Table E

2 2-way ANOVACh 21

Table E

No

START HERE

Chapter 20: Chi-Square (χ2 ) and Inference about Frequencies

Example: Research suggests that 13% of the population is left handed. Of the 91 students who took our survey, 7 indicated that they were left-handed. Are the numbers we observer for our class significantly different than what is expected from the population?

A Chi-Square Test (χ2) for Goodness of Fit tests hypotheses about the number of cases falling into the categories of a frequency distribution.

A χ2 test is a hypothesis test on categorical data.

Right LeftObserved (fo) 84 7

Expected (fe) 91*(1-.13) =79.17 91*.13 = 11.83

Note: we’re skipping sections 20.11 and 20.12 in the book (effect size and power)

The number χ2 (‘Chi-Square”) is a statistic that measures how close our observed frequencies (fo) are to the expected frequencies (fe).

e

eo

fff 2

2

26.229.097.183.11

)83.117(17.79

)17.7984( 222

You can see that χ2 should increase with the discrepancy between the observed and expected frequencies.

Right LeftObserved (fo) 84 7

Expected (fe) 79.17 11.83

The shape of the χ2 distribution depends on the number of categories in the frequency distribution. For this example, there are two categories (‘left handed’ and ‘right handed’).

Tables for χ2 are defined by their degrees of freedom (df). For examples like this, df = n-1, where n is the number of categories (not the number of measurements within or across each category).

Table I provides critical values for χ2 for a given area in the tail and df.

For this example, suppose we choose a = .05.

Table I says that the critical value of χ2 for df = 1 and a = .05 is 3.84

Since our observed value of χ2 is 2.26, we fail to reject H0 and conclude that the frequency of left (or right) handers in our class is not significantly different than the population.

What does the χ2 distribution look like?

It varies a lot with df. Here’s what the χ2 distribution looks like for df = 1:

0 2 4 6 8 102

Rel

ativ

e Fr

eque

ncy

Pr(χ2>3.84) = .05

Right LeftObserved (fo) 84 7

Anyone notice that we already had a way of testing this hypothesis?

We could have used the normal approximation to the binomial, with n= 91 and P = .13

21.3)87)(.13)(.91(

83.11)13)(.91(

nPQ

nP

s

m

505.121.3

83.117

smXz

26.22 We calculated that

note that: 26.2)505.1( 22 z

It turns out that for two categories (df = 1) , 22 z

e

eo

fff 2

2

In general, a 2 distribution with k=df degrees of freedom is the probability distribution for drawing k scores from a standard normal distribution, squaring them, and adding them up.

Something for the geeks out there…

Example: The birthdays of the 91 students in this class are distributed across the months as follows:

Is this distribution of students significantly different than what would be expected if there was an equal probability of birthday for each month? (use a = .05)

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

f0 8 8 11 11 8 7 5 11 2 9 8 3

Here’s a plot of the frequency distribution of the data:

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec0

2

4

6

8

10

12

Freq

uenc

y

Birthday Month

Example: The birthdays of the 91 students in this class are distributed across the months as follows:

Is this distribution of students significantly different than what would be expected if there was an equal probability of birthday for each month? (use a = .05)

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

fe 7.58 7.58 7.58 7.58 7.58 7.58 7.58 7.58 7.58 7.58 7.58 7.58

Answer: The null hypothesis (H0) is that there is even distribution of birthdays across the months. The alternative hypothesis HA is that the birthdays are not evenly distributed. Under H0, the expected number (fe) of students with birthdays for each month would be 91/12 = 7.58

Testing H0 requires a χ2 test with df = 12-1 = 11.

Using Table I with a = .05 and df = 11, the critical value of χ2 is 19.68

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

f0 8 8 11 11 8 7 5 11 2 9 8 3

77.1258.7

)58.73(58.7

)58.78(58.7

)58.78( 22222

e

eo

fff

0 5 10 15 20 25 30

2

Rel

ativ

e Fr

eque

ncy

2 distribution with df = 11Our observed value of χ2 (12.77) is less than the critical value of χ2 (19.68).

Our decision is to fail to reject H0 and conclude that the distribution of birthdays in this class is not significantly different from what is expected from an even distribution.

Pr(χ2>19.68) = .05

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

f0 8 8 11 11 8 7 5 11 2 9 8 3

fe 7.58 7.5 7.58 7.58 7.58 7.58 7.58 7.58 7.58 7.58 7.58 7.58

Another type of χ2 test is to test for independence between two variables.

Apple PC

Female 39 31

Male 8 12

Example: For the students in this class that use either an Apple or a PC, the distribution of computer users across genders is as follows:

Does this distribution show a significant dependence between gender and computer type? In other words, does the choice of computers differ significantly across genders? (Use a = .05)

We can visualize our frequencies with a bar graph:

Apple PC

Female 39 31

Male 8 12

To begin, we need to set up the expected frequencies under the null hypothesis (H0) that there is not a dependence of computer type on gender.

This is done by computing the ‘marginal totals’ for each row and column.

The expected frequency for each cell is the product of the cell’s row total and the cell’s column total, divided by the grand total:

Apple PC frow

Female 70

Male 27

fcol 33/97 = .3402 64/97 = .6598 Total =97

Apple PC frow

Female (70)(47)/90 = 36.56 (70)(43)/90 = 33.44 70

Male (20)(47)/90 = 10.44 (20)(47)/90 = 10.44 20

fcol 47 43 Total =90

Apple PC frow

Female 39 31 70

Male 8 12 20fcol 47 43 Total =90

So our expected frequencies (fe) are:

And (again) our observed frequencies (fo) are:

1.53 = 0.62 + 0.18 + 0.57 + 0.16

56.9

)56.912(44.33

)44.3331(44.10

)44.108(56.36

)56.3639( 222222

e

eo

fff

The χ2 statistic is calculated as before by comparing fe to fo:

Apple PC

Female 36.56 33.44

Male 10.44 9.56

Apple PC

Female 39 31

Male 8 12

53.12

2

e

eo

fff

The degrees of freedom is equal to (C-1) x (R-1), where C and R are the numbers of rows and columns of our matrix.

For our example, C = 2, and R = 2 so df = (2-1) x (2-1) = 1

Recall from the first example that the value of χ2crit for df = 1 and a = .05 is 3.84

Our observed value of χ2 (1.53) is less than the critical value of χ2 (3.84). We therefore fail to reject H0 and conclude that there is not a significant dependence of computer choice on gender.

Apple PC

Female 39 31

Male 8 12

Example: 94 students in our class responded to the question if they played video games ‘not at all’, ‘just a little’, ‘a fair amount’ and ‘very much’. The breakdown of the responses by gender is as follows:

Is there a significant relationship between gender and amount of video game playing? Use a = .01.

fo Not at all Just a little A fair amount Very much

Female 27 26 14 6

Male 1 5 8 7

Answer: We will test the null hypothesis that video game playing and gender are independent.

Female Male0

5

10

15

20

25

30

Freq

uenc

y

Not at allJust a littleA fair amountVery much

fo Not at all Just a little A fair amount Very much

Female 27 26 14 6

Male 1 5 8 7

Video Game Playing

fe Not at all Just a little

A fair amount

Very much Totals

Female 21.7 24.1 17.1 10.1 73

Male 6.3 6.9 4.9 2.9 21

Totals 28 31 22 13 94

Fill in the expected frequencies. For example, for ‘Female’ and ‘not at all’:(73)(28)/94 = 21.7

fo Not at all Just a little A fair amount Very much

Female 27 26 14 6

Male 1 5 8 7

4.169.2

)9.27(...1.24

1.24267.21

7.2127 22222

e

eo

fff

df = (R-1)x(C-1) = (4-1)x(2-1) = 3

χ2crit for df = 3, and a = .01 is 11.35. Since our observed value of χ2 is greater than χ2

crit, we reject H0 and conclude that the amount of video game playing does depend on gender.

fo Not at all Just a little A fair amount Very much

Female 27 26 14 6

Male 1 5 8 7

fe Not at all Just a little A fair amount Very much

Female 21.7 24.1 17.1 10.1

Male 6.3 6.9 4.9 2.9

fo Blue Green Orange Pink Purple Red Yellow

Female 15 11 2 8 20 10 9

Male 4 8 2 0 0 7 0

Use a χ2 test to determine if student’s favorite color depends on gender (use a = .01).

(formally, test the null hypothesis that there is no relation between the color and gender)

Favorite color

Gend

er

fo Blue Green Orange Pink Purple Red Yellow

Female 15 11 2 8 20 10 9

Male 4 8 2 0 0 7 0

Female Male0

5

10

15

20

Freq

uenc

y

Favorite Color

fo Blue Green Orange Pink Purple Red Yellow Totals

Female 15 11 2 8 20 10 9 75

Male 4 8 2 0 0 7 0 21

Totals 19 19 4 8 20 17 9 96

fe Blue Green Orange Pink Purple Red Yellow Totals

Female 14.8 14.8 3.1 6.3 15.6 13.3 7 75

Male 4.2 4.2 0.9 1.8 4.4 3.7 2 21

Totals 19 19 4 8 20 17 9 96

χ2crit = 16.81 (df =6)χ2

obs = 20.39

We reject H0: There is a relation between gender and preferred color (p-value is .0024)

fo Blue Green Orange Pink Purple Red Yellow

Female 15 11 2 8 20 10 9

Male 4 8 2 0 0 7 0

fe Blue Green Orange Pink Purple Red Yellow

Female 14.8 14.8 3.1 6.3 15.6 13.3 7

Male 4.2 4.2 0.9 1.8 4.4 3.7 2