2.5 Hypothesis Testing - QMUL Mathsbb/MS_NotesWeek12.pdf · HYPOTHESIS TESTING 115 Example 2.33....

16

Click here to load reader

Transcript of 2.5 Hypothesis Testing - QMUL Mathsbb/MS_NotesWeek12.pdf · HYPOTHESIS TESTING 115 Example 2.33....

Page 1: 2.5 Hypothesis Testing - QMUL Mathsbb/MS_NotesWeek12.pdf · HYPOTHESIS TESTING 115 Example 2.33. Examples of ... is true and a type II error is to ... same significance level and

114 CHAPTER 2. ELEMENTS OF STATISTICAL INFERENCE

2.5 Hypothesis Testing

We assume thatY1, . . . , Yn have a joint distribution which depends on the un-known parametersϑ = (ϑ1, . . . , ϑp)

T. The set of all possible values ofϑ is calledtheparameter spaceΘ.

Example2.31. Parameter spaces for various distributions:

1. Yi ∼iid

Bin(m, p), wherem is known, i = 1, 2, . . . , n. Thenϑ = p and

Θ = (0, 1).

2. Yi ∼iid

Poisson(λ), i = 1, 2, . . . , n. Thenϑ = λ andΘ = R+.

3. Yi ∼iid

N (µ, σ2), i = 1, 2, . . . , n. Thenϑ = (µ, σ2)T andΘ = R× R+.

4. Yi ∼ N (β0 + β1xi, σ2) independently fori = 1, 2, . . . , n. Thenϑ =

(β0, β1, σ2)T andΘ = R

2 × R+.

A hypothesis restrictsϑ to lie in Θ?, whereΘ? ⊂ Θ. If we wish to test whetherϑ ∈ Θ?, then we test thenull hypothesisH0 : ϑ ∈ Θ? against thealternativehypothesisH1 : ϑ ∈ Θ\Θ?.

Example2.32. Examples of setsΘ? restricted by null hypotheses:

1. Yi ∼iid

Poisson(λ), i = 1, 2, . . . , n. If we testH0 : λ = λ0 againstH1 : λ 6=λ0 thenΘ? = {λ0}.

2. Yi ∼iid

N (µ, σ2), i = 1, 2, . . . , n. If we testH0 : µ = µ0 againstH1 : µ 6= µ0

thenΘ? = {µ0} × R+.

3. Yi ∼ N (β0 + β1xi, σ2) independently fori = 1, 2, . . . , n. For testingH0 :

β1 = 0 againstH1 : β1 6= 0 we haveΘ? = R× {0} × R+.

If Θ? is a single point, the hypothesis issimple. Otherwise, the hypothesis iscomposite.

The set of all possible valuesy of the random sampleY is called thesamplespaceY .

Page 2: 2.5 Hypothesis Testing - QMUL Mathsbb/MS_NotesWeek12.pdf · HYPOTHESIS TESTING 115 Example 2.33. Examples of ... is true and a type II error is to ... same significance level and

2.5. HYPOTHESIS TESTING 115

Example2.33. Examples of sample spaces:

1. Yi ∼iid

Poisson(λ), i = 1, 2, . . . , n. ThenY = [Z+ ∪ {0}]n.

2. Yi ∼iid

N (µ, σ2), i = 1, 2, . . . , n. ThenY = Rn.

Therejection region is the subsetR ⊆ Y such that we rejectH0 if y ∈ R.

2.5.1 Type I and II errors

There are two errors in hypothesis testing. Atype I error is to rejectH0 whenH0

is true and atype II error is to acceptH0 whenH0 is false.

WhenH0 is simple, that is, we haveH0 : ϑ = ϑ0, thesignificance levelor sizeof the test is defined to be

α = P (type I error) = P (Y ∈ R|H0 true) = P (Y ∈ R|ϑ = ϑ0).

WhenH0 is composite, that is, we have

H0 : ϑ ∈ Θ?,

whereΘ? is not a single point, then we have a significance functionα(ϑ) =P (Y ∈ R|ϑ) which depends on the particular value inΘ? taken byϑ. In thiscase, thesizeof the test orsignificance levelis defined to be

α = maxϑ∈Θ?

α(ϑ).

WhenH1 is simple, that is, we haveH1 : ϑ = ϑ1, thepower of the test is definedto be

β = 1−P (type II error) = 1−P (Y ∈ Rc|H1 true) = P (Y ∈ R|H1 true) = P (Y ∈ R|ϑ = ϑ1),

whereRc is a complement ofR (not a rejection region).

Page 3: 2.5 Hypothesis Testing - QMUL Mathsbb/MS_NotesWeek12.pdf · HYPOTHESIS TESTING 115 Example 2.33. Examples of ... is true and a type II error is to ... same significance level and

116 CHAPTER 2. ELEMENTS OF STATISTICAL INFERENCE

WhenH1 is composite, that is, we have

H1 : ϑ ∈ Θ\Θ?,

then we have apower function

β(ϑ) = P (Y ∈ R|ϑ)

which depends on the value inΘ\Θ? taken byϑ.

2.5.2 Neyman-Pearson lemma

Suppose thatH0 andH1 are both simple hypotheses, so that we wish to testH0 :ϑ = ϑ0 againstH1 : ϑ = ϑ1. Then we can compare many different tests withthe same significance levelα. We will choose the one with the largest power,that is, for a fixedα = P (type I error) we choose the test that maximizesβ =1− P (type II error). The following famous result gives a way of doing this.

Lemma 2.4. (Neyman-Pearson)LetY1, . . . , Yn have a joint distribution depending on the parametersϑ and con-sider testingH0 : ϑ = ϑ0 againstH1 : ϑ = ϑ1. Then the test with the highestpower among all tests with significance levelα has rejection region

R = {y : λ(y) ≤ a},

where

λ(y) =L(ϑ0|y)L(ϑ1|y)

is thelikelihood ratio anda is a constant determined byα, that isa such that

P (λ(Y ) ≤ a|H0) = α.

Proof. We prove the result in the continuous case only. The proof in the discretecase is similar.

Consider any other test with significance levelα. Denote its rejection region byR′. Further, denote the powers of the tests based onR andR′ by β and β ′,respectively. Then, in order to prove the required result, we need only show thatβ ′ < β.

Page 4: 2.5 Hypothesis Testing - QMUL Mathsbb/MS_NotesWeek12.pdf · HYPOTHESIS TESTING 115 Example 2.33. Examples of ... is true and a type II error is to ... same significance level and

2.5. HYPOTHESIS TESTING 117

Let B = R ∩ R′. Then, since the significance level for both tests isα, we maywrite

α = P (Y ∈ R|H0 true) =∫

. . .

B

f(y;ϑ0)dy1 . . . dyn+

∫. . .

R\B

f(y;ϑ0)dy1 . . . dyn

and

α = P (Y ∈ R′|H0 true) =∫

. . .

B

f(y;ϑ0)dy1 . . . dyn+

∫. . .

R′\B

f(y;ϑ0)dy1 . . . dyn.

It follows that∫

. . .

R\B

f(y;ϑ0)dy1 . . . dyn =

∫. . .

R′\B

f(y;ϑ)dy1 . . . dyn.

Now, fory ∈ R, and hence fory ∈ R\B, we know thatλ(y) ≤ a, so that

L(ϑ0|y) ≤ aL(ϑ1|y).

Also, fory 6∈ R, and hence fory ∈ R′\B, we know thatλ(y) > a, so that

L(ϑ0|y) > aL(ϑ1|y).

These inequalities also hold for the joint pd functionsf , therefore,

β ′ = P (Y ∈ R′|H1 true)

=

∫. . .

B

f(y;ϑ1)dy1 . . . dyn +

∫. . .

R′\B

f(y;ϑ1)dy1 . . . dyn

<

∫. . .

B

f(y;ϑ1)dy1 . . . dyn +1

a

∫. . .

R′\B

f(y;ϑ0)dy1 . . . dyn

=

∫. . .

B

f(y;ϑ1)dy1 . . . dyn +1

a

∫. . .

R\B

f(y;ϑ0)dy1 . . . dyn

≤∫

. . .

B

f(y;ϑ1)dy1 . . . dyn +1

aa

∫. . .

R\B

f(y;ϑ1)dy1 . . . dyn

=

∫. . .

B

f(y;ϑ1)dy1 . . . dyn +

∫. . .

R\B

f(y;ϑ1)dy1 . . . dyn

= P (Y ∈ R|H1 true) = β,

which completes the proof. �

Page 5: 2.5 Hypothesis Testing - QMUL Mathsbb/MS_NotesWeek12.pdf · HYPOTHESIS TESTING 115 Example 2.33. Examples of ... is true and a type II error is to ... same significance level and

118 CHAPTER 2. ELEMENTS OF STATISTICAL INFERENCE

Example2.34. Suppose thatY1, . . . , Yn are independentPoisson(λ) random vari-ables and consider testingH0 : λ = λ0 againstH1 : λ = λ1, whereλ1 > λ0.

Then the likelihood is

L(λ|y) = λ∑

n

i=1yie−nλ

∏ni=1 yi!

,

and so the likelihood ratio is

λ(y) =L(λ0|y)L(λ1|y)

=λ∑

n

i=1yi

0 e−nλ0

∏ni=1 yi!

∏ni=1 yi!

λ∑

n

i=1yi

1 e−nλ1

=

(λ0

λ1

)∑n

i=1yi

en(λ1−λ0).

Since the rejection region is

R = {y : λ(y) ≤ a},

we rejectH0 if (λ0

λ1

)∑n

i=1yi

en(λ1−λ0) ≤ a,

wherea is a constant chosen to give significance levelα, thata is such that

P

((λ0

λ1

)∑n

i=1yi

en(λ1−λ0) ≤ a|H0

)= α.

Thus, rearranging, we rejectH0 if

(λ0

λ1

)∑n

i=1yi

≤ b ⇒n∑

i=1

yi log

(λ0

λ1

)≤ c,

whereb andc are constants chosen to give significance levelα.

It follows that, sinceλ1 > λ0, the rejection region is of the form

R = {y : y ≥ d},

whered is a constant chosen to give significance levelα.

To findd, we must work out the sampling distribution ofY and choosed such thatP (Y ≥ d|λ = λ0) = α. This is difficult, but the central limit theorem tells usthat, for largen,

Y ∼approx.

N(λ,

λ

n

).

Page 6: 2.5 Hypothesis Testing - QMUL Mathsbb/MS_NotesWeek12.pdf · HYPOTHESIS TESTING 115 Example 2.33. Examples of ... is true and a type II error is to ... same significance level and

2.5. HYPOTHESIS TESTING 119

Thus, since

P (Y ≥ d|λ = λ0) = α ⇒ P

(Y − λ0√λ0/n

≥ d− λ0√λ0/n

|λ = λ0

)= α

and sinceY − λ0√λ0/n

∼approx.

N (0, 1)

for largen whenH0 is true, we have

Φ

(d− λ0√λ0/n

)= 1− α ⇒ d− λ0√

λ0/n= zα.

So we rejectH0 if, for a given data sety we have

y ≥ λ0 + zα√λ0/n.

Similarly, if λ1 < λ0, we rejectH0 if

y ≤ λ0 − zα√

λ0/n.

Now suppose thatH0 is simple and thatH1 is composite, so that we wish to testH0 : ϑ = ϑ0 againstH1 : ϑ ∈ Θ\{ϑ0}. Then we again compare tests with thesame significance level and try to obtain one which has a powerfunction greaterthan or equal to the power function of any other test at all values ofϑ ∈ Θ\{ϑ0}.Such a test is called auniformly most powerful (UMP) test.

We find the most powerful test ofH0 : ϑ = ϑ0 againstH1 : ϑ = ϑ1 and obtainthe rejection regionR.

If the same rejection regionR is obtained for allϑ1 ∈ Θ\{ϑ0}, then there is aUMP test and its critical region isR.

If more than one rejection region is found, then there is no UMP test.

Example2.35. Suppose thatY1, . . . , Yn are independentPoisson(λ) random vari-ables and consider testingH0 : λ = λ0 againstH1 : λ > λ0. Then we saw that forH1 : λ = λ1, the rejection region is

R =

{y : y ≥ λ0 + zα

√λ0

n

}.

Page 7: 2.5 Hypothesis Testing - QMUL Mathsbb/MS_NotesWeek12.pdf · HYPOTHESIS TESTING 115 Example 2.33. Examples of ... is true and a type II error is to ... same significance level and

120 CHAPTER 2. ELEMENTS OF STATISTICAL INFERENCE

Since this does not depend onλ1, it is the same for eachλ1 > λ0 and gives a UMPtest. Forλ > λ0, the power function of the test is given by

β(λ) = P (Y ∈ R|λ) = P

(Y ≥ λ0 + zα

√λ0

n|λ)

= P

(Y − λ√λ/n

≥ λ0 + zα√

λ0/n− λ√λ/n

|λ)

' 1− Φ

(λ0 + zα

√λ0/n− λ√λ/n

)

= 1− Φ{g(λ)},

sinceY ∼ N (λ, λ/n) approximately for largen.

To testH0 : λ = λ0 againstH1 : λ < λ0, we similarly obtain a UMP test basedon

R =

{y : y ≤ λ0 − zα

√λ0

n

}.

However, to testH0 : λ = λ0 againstH1 : λ 6= λ0, there is no UMP test, sinceRis different forλ1 > λ0 andλ1 < λ0. �

The above is typical. We can usually find UMP tests for one-sided alternatives,but not for two-sided ones.

Example2.36. An observation of a traffic in a mountain tunnel was carried outover a 100 minutes randomly chosen in several days. It gave anestimate of themean number of cars entering the tunnel per minute equal toλ̂ = y = 1.3. Morethen one car per minute may cause a hazardous situation in thetunnel. Test thenull hypothesisH0 : λ = 1 againstH1 : λ > 1. Give the power function whenα = 0.05.

The rejection region is

R =

{y : y ≥ λ0 + zα

√λ0

n

}=

{y : y ≥ 1 + 1.6449

√1

100

}= {y : y ≥ 1.16449}

We have observedy = 1.3 ∈ R, hence, at the significance levelα = 0.05, werejectH0 : λ = 1 againstH1 : λ > 1. The traffic in the tunnel is hazardous.

Page 8: 2.5 Hypothesis Testing - QMUL Mathsbb/MS_NotesWeek12.pdf · HYPOTHESIS TESTING 115 Example 2.33. Examples of ... is true and a type II error is to ... same significance level and

2.5. HYPOTHESIS TESTING 121

The power function is

β(λ) = 1− Φ

(λ0 + zα

√λ0/n− λ√λ/n

)= 1− Φ{g(λ)},

whereg(λ) =λ0+zα

√λ0/n−λ√

λ/n. Note that,g(λ) → −∞ whenλ → ∞. That is,

Φ{g(λ)} → 0 and so the power functionβ(λ) → 1 whenλ → ∞. It means thatthe farther isλ from the assumedλ0 in the null hypothesis, the larger is the powerof the test, i.e., the probability to reject the null hypothesis when the alternative istrue.

1.0 1.5 2.0 2.5 3.0 λ

0.0

0.2

0.4

0.6

0.8

1.0

Pow

er o

f the

test

β(λ

)

Fig. 1. Power Function for the testH0 : λ = 1 againstH1 : λ > 1.

Exercise2.17. Suppose thatYi ∼iid

Bernoulli(p), i = 1, . . . , n. Consider testing

the hypothesisH0 : p = p0 againstH1 : p = p1.

(a) Use the Neyman-Pearson lemma to obtain the general form of the rejectionregion of the most powerful test ofH0 againstH1. For large samples, findthe form of the rejection region for the test at a significancelevelα.

(b) Show that a uniformly most powerful test ofH0 : p = p0 againstH1 : p > p0exists and obtain the power function for the test.

Page 9: 2.5 Hypothesis Testing - QMUL Mathsbb/MS_NotesWeek12.pdf · HYPOTHESIS TESTING 115 Example 2.33. Examples of ... is true and a type II error is to ... same significance level and

122 CHAPTER 2. ELEMENTS OF STATISTICAL INFERENCE

Exercise2.18. When in pre-clinical studies a candidate drug is tested for safety, itis often done on animals. Assume that an experimenter expects that a proportionpof toxic responses to a new medicinal product is 0.1. This level of toxicity wouldbe acceptable to continue the studies in phase I of clinical trials where it is firsttested on humans. The experimenter wants to conduct an experiment onn mice.After applying the drug he will observe whether the responseof each mouse showssymptoms of toxicity or not. He wants to test the hypothesis

H0 : p = 0.1against H1 : p > 0.1

It can be assumed that all the responses are independent.

(a) Obtain a rejection region to test this hypothesis at the significance levelα =0.05. Is there evidence to reject the null hypothesis ifn = 30 mice wereused in the experiment and six showed toxic responses?

(b) Calculate the power of the test

H0 : p = 0.1against H1 : p = 0.2

whenn = 30 andα = 0.05.

How many mice would be needed in the experiment to obtain the power ofthe test equal to 0.8? What are your conclusions?

2.5.3 Generalized likelihood ratio tests

When a UMP test does not exist, we usually use a generalized likelihood ratiotest to verifyH0 : ϑ ∈ Θ? againstH1 : ϑ ∈ Θ\Θ?. It can be used whenH0 iscomposite, which none of the above methods can.

The generalized likelihood ratio test has rejection regionR = {y : λ(y) ≤ a},where

λ(y) =maxϑ∈Θ? L(ϑ|y)maxϑ∈Θ L(ϑ|y)

is thegeneralized likelihood ratioanda is a constant chosen to give significancelevelα, that is such that

P (λ(Y ) ≤ a|H0) = α.

Page 10: 2.5 Hypothesis Testing - QMUL Mathsbb/MS_NotesWeek12.pdf · HYPOTHESIS TESTING 115 Example 2.33. Examples of ... is true and a type II error is to ... same significance level and

2.5. HYPOTHESIS TESTING 123

If we let ϑ̂ denote the maximum likelihood estimate ofϑ and letϑ̂0 denote thevalue ofϑ which maximises the likelihood over all values ofϑ in Θ?, then wemay write

λ(y) =L(ϑ̂0|y)L(ϑ̂|y)

.

The quantityϑ̂0 is called therestricted maximum likelihood estimateofϑ underH0.

Example2.37. Suppose thatYi ∼iid

N (µ, σ2) and consider testingH0 : µ = µ0

againstH1 : µ 6= µ0. Then the likelihood is

L(µ, σ2|y) = (2πσ2)−n

2 exp

{− 1

2σ2

n∑

i=1

(yi − µ)2

}.

The maximum likelihood estimate ofϑ = (µ, σ2)T is ϑ̂ = (µ̂, σ̂2)T, whereµ̂ = yandσ̂2 =

∑ni=1(yi − y)2/n.

Similarly, the restricted maximum likelihood estimate ofϑ = (µ, σ2)T underH0

is ϑ̂0 = (µ̂0, σ̂20)

T, whereµ̂0 = µ0 andσ̂20 =

∑ni=1(yi − µ0)

2/n.

Thus, the generalized likelihood ratio is

λ(y) =L(µ0, σ̂

20|y)

L(µ̂, σ̂2|y) =(2πσ̂2

0)−n

2 exp{− 1

2σ̂2

0

∑ni=1(yi − µ0)

2}

(2πσ̂2)−n

2 exp{− 1

2σ̂2

∑ni=1(yi − y)2

}

=

(σ̂2

σ̂20

)n

2

exp

{n∑n

i=1(yi − y)2

2∑n

i=1(yi − y)2− n

∑ni=1(yi − µ0)

2

2∑n

i=1(yi − µ0)2

}

=

{ ∑ni=1(yi − y)2∑ni=1(yi − µ0)2

}n

2

.

Since the rejection region isR = {y : λ(y) ≤ a}, we rejectH0 if

{ ∑ni=1(yi − y)2∑ni=1(yi − µ0)2

}n

2

≤ a ⇒∑n

i=1(yi − y)2∑ni=1(yi − µ0)2

≤ b,

wherea andb are constants chosen to give significance levelα. Now, we may

Page 11: 2.5 Hypothesis Testing - QMUL Mathsbb/MS_NotesWeek12.pdf · HYPOTHESIS TESTING 115 Example 2.33. Examples of ... is true and a type II error is to ... same significance level and

124 CHAPTER 2. ELEMENTS OF STATISTICAL INFERENCE

write

n∑

i=1

(yi − µ0)2 =

n∑

i=1

{(yi − y) + (y − µ0)}2

=

n∑

i=1

(yi − y)2 + 2(y − µ0)

n∑

i=1

(yi − y) + n(y − µ0)2

=n∑

i=1

(yi − y)2 + n(y − µ0)2.

So we rejectH0 if ∑ni=1(yi − y)2∑n

i=1(yi − y)2 + n(y − µ0)2≤ b.

Thus, rearranging, we rejectH0 if

1 +n(y − µ0)

2

∑ni=1(yi − y)2

≥ c ⇒ n(y − µ0)2

1n−1

∑ni=1(yi − y)2

≥ d,

wherec andd are constants chosen to give significance levelα, that is we canwrite

α = P (λ(Y ) ≤ a|H0) = P

(n(Y − µ0)

2

S2≥ d|H0

),

whereS2 = 1n−1

∑ni=1(Yi − Y )2.

To getd we need to work out the distribution ofn(Y−µ0)2

S2 under the null hypothesis.ForYi ∼

iidN (µ, σ2) we have

Y ∼ N

(µ,

σ2

n

)

and so, underH0,

Y ∼ N(µ0,

σ2

n

)and

√n(Y − µ0

)√σ2

∼ N (0, 1).

This givesn(Y − µ0)

2

σ2∼ χ2

1.

Also(n− 1)S2

σ2∼ χ2

n−1.

Page 12: 2.5 Hypothesis Testing - QMUL Mathsbb/MS_NotesWeek12.pdf · HYPOTHESIS TESTING 115 Example 2.33. Examples of ... is true and a type II error is to ... same significance level and

2.5. HYPOTHESIS TESTING 125

Now we may use the fact that ifU andV are independent rvs such thatU ∼ χ2ν1

andV ∼ χ2ν2 , thenU/ν1

V/ν2∼ Fν1,ν2.

Here,U = n(Y−µ0)2

σ2 andV = (n−1)S2

σ2 . Hence, ifH0 is true, we have

F =U/1

V/(n− 1)=

n(Y − µ0)2

S2∼ F1,n−1.

Therefore, we rejectH0 at a significance levelα if

n(y − µ0)2

s2≥ F1,n−1,α,

whereF1,n−1,α is such thatP (F ≥ F1,n−1,α) = α.

Equivalently, we rejectH0 if√

n(y − µ0)2

s2≥ tn−1,α

2,

that is, if ∣∣∣∣∣y − µ0√s2/n

∣∣∣∣∣ ≥ tn−1,α2.

Of course, this is the usual two-sidedt test.�

In fact, many of the standard tests in situations with normaldistributions are gen-eralized likelihood ratio tests.

2.5.4 Wilks’ theorem

In more complex cases, we have to use the following approximation to find therejection region. The result is stated without proof.

Theorem 2.9.Wilks’ theorem.Assume that the joint distribution ofY1, . . . , Yn depends onp unknown parametersand that, underH0, the joint distribution depends onp0 unknown parameters. Letν = p − p0. Then, under some regularity conditions, when the null hypothesis is

Page 13: 2.5 Hypothesis Testing - QMUL Mathsbb/MS_NotesWeek12.pdf · HYPOTHESIS TESTING 115 Example 2.33. Examples of ... is true and a type II error is to ... same significance level and

126 CHAPTER 2. ELEMENTS OF STATISTICAL INFERENCE

true, the distribution of the statistic−2 log{λ(Y )} converges to aχ2ν distribution

as the sample sizen → ∞, i.e., whenH0 is true andn is large,

−2 log{λ(Y )} ∼approx.

χ2ν .

Thus, for largen, the rejection region for a test with approximate significancelevelα is

R = {y : −2 log{λ(y)} ≥ χ2ν,α}.

Example2.38. Suppose thatYi ∼iid

Poisson(λ) and consider testingH0 : λ = λ0

againstH1 : λ 6= λ0.

Then we have seen that no UMP test exists in this case. Now, thelikelihood is

L(λ|y) = λ∑

n

i=1yie−nλ

∏ni=1 yi!

.

The maximum likelihood estimate ofλ is λ̂ = y and the restricted maximumlikelihood estimate ofλ underH0 is λ̂0 = λ0. Thus, the generalized likelihoodratio is

λ(y) =L(λ0|y)L(λ̂|y)

=λ∑

n

i=1yi

0 e−nλ0

∏ni=1 yi!

∏ni=1 yi!

y∑

n

i=1yie−ny

=

(λ0

y

)∑n

i=1yi

en(y−λ0).

It follows that

−2 log{λ(y)} = −2

{ny log

(λ0

y

)+ n(y − λ0)

}

= 2n

{y log

(y

λ0

)+ λ0 − y

}.

Here,p = 1 andp0 = 0, and soν = 1. Therefore, by Wilks’ theorem, whenH0 istrue andn is large,

2n

{Y log

(Y

λ0

)+ λ0 − Y

}∼ χ2

1.

Page 14: 2.5 Hypothesis Testing - QMUL Mathsbb/MS_NotesWeek12.pdf · HYPOTHESIS TESTING 115 Example 2.33. Examples of ... is true and a type II error is to ... same significance level and

2.5. HYPOTHESIS TESTING 127

Hence, for a test with approximate significance levelα, we rejectH0 if and onlyif

2n

{y log

(y

λ0

)+ λ0 − y

}≥ χ2

1,α.

Example2.39. Suppose thatYi, i = 1, . . . , n, are iid random variables with theprobability mass function given by

P (Y = y) =

{ϑj , if y = j, j = 1, 2, 3;0, otherwise,

whereϑj are unknown parameters such thatϑ1+ϑ2+ϑ3 = 1 andϑj ≥ 0. Considertesting

H0 : ϑ1 = ϑ2 = ϑ3

against H1 : H0 is not true

We will use the Wilks’ theorem to derive the critical region for testing this hy-pothesis at an approximate significance levelα.

Here the full parameter spaceΘ is two-dimensional because there are only twofree parameters, i.e.,ϑ3 = 1− ϑ1 − ϑ2 and

Θ = {ϑ = (ϑ1, ϑ2, ϑ3)T : ϑ3 = 1− ϑ1 − ϑ2, ϑj ≥ 0}.

Hencep = 2.

The restricted parameter space is zero-dimensional, because under the null hy-pothesis all the parameters are equal and as they sum up to 1, they all must beequal to1/3. Hence

Θ? =

{ϑ =

(1

3,1

3,1

3

)T}

and sop0 = 0 (zero unknown parameters). That is the number of degrees offreedom of theχ2 distribution isν = p− p0 = 2.

To calculateλ(Y ) we need to find the MLE(ϑ) inΘ and in the restricted spaceΘ?.

MLE (ϑ) in Θ:The likelihood function is

L(ϑ;y) = ϑn1

1 ϑn2

2 (1− ϑ1 − ϑ2)n3 ,

Page 15: 2.5 Hypothesis Testing - QMUL Mathsbb/MS_NotesWeek12.pdf · HYPOTHESIS TESTING 115 Example 2.33. Examples of ... is true and a type II error is to ... same significance level and

128 CHAPTER 2. ELEMENTS OF STATISTICAL INFERENCE

wherenj is the number of responses equal toj, j = 1, 2, 3. Then, the log-likelihood is

l(ϑ;y) = n1 log(ϑ1) + n2 log(ϑ2) + n3 log(1− ϑ1 − ϑ2).

For j = 1, 2 we have,

∂l

∂ϑj=

nj

ϑj+

n3

1− ϑ1 − ϑ2(−1).

When compared to zero, this gives

ϑ̂j

nj=

ϑ̂3

n3= γ.

Now, sinceϑ̂1 + ϑ̂2 + ϑ̂3 = 1 we obtainγ = 1n, wheren = n1 + n2 + n3. Then

the estimates of the parameters areϑ̂j = nj/n, j = 1, 2, 3.

The second derivatives are

∂2l

∂ϑ2j

= −nj

ϑ2j

− n3

ϑ23

∂2l

∂ϑjϑi= −n3

ϑ23

The determinant of the second derivatives is positive for all ϑ and the element∂2l

∂ϑ2

1

is negative for allϑ, so also forϑ̂. Hence, there is a maximum atϑ̂ and

MLE(ϑj) = ϑ̂j =nj

n.

MLE(ϑ) in Θ?:

L(ϑ;y) =

(1

3

)n1(1

3

)n2(1

3

)n3

.

That isϑ̂0 =(13, 13, 13

)T. Now, we can calculateλ(y):

λ(y) =L(ϑ̂0;y)

L(ϑ̂;y)=

(13

)n1(13

)n2(13

)n3

(n1

n

)n1(n2

n

)n2(n3

n

)n3=

(n

3n1

)n1(

n

3n2

)n2(

n

3n3

)n3

.

That is

−2 log{λ(y)} = −23∑

j=1

nj log

(n

3nj

).

Page 16: 2.5 Hypothesis Testing - QMUL Mathsbb/MS_NotesWeek12.pdf · HYPOTHESIS TESTING 115 Example 2.33. Examples of ... is true and a type II error is to ... same significance level and

2.5. HYPOTHESIS TESTING 129

We rejectH0 at an approximate significance levelα if the observed sample be-longs to the critical regionR, where

R =

{y : 2

3∑

j=1

nj log

(3nj

n

)≥ χ2

2;α

}.