Classical Hypothesis Testing Theory Alexander Senf

Click here to load reader

  • date post

    21-Dec-2015
  • Category

    Documents

  • view

    214
  • download

    1

Embed Size (px)

Transcript of Classical Hypothesis Testing Theory Alexander Senf

  • Slide 1
  • Classical Hypothesis Testing Theory Alexander Senf
  • Slide 2
  • Review 5 steps of classical hypothesis testing (Ch. 3) 1.Declare null hypothesis H 0 and alternate hypothesis H 1 2.Fix a threshold for Type I error (1% or 5%) Type I error (): reject H 0 when it is true Type II error (): accept H 0 when it is false 3.Determine a test statistic a quantity calculated from the data 27/31/2008
  • Slide 3
  • Review 4.Determine what observed values of the test statistic should lead to rejection of H 0 Significance point K (determined by ) 5.Test to see if observed data is more extreme than significance point K If it is, reject H 0 Otherwise, accept H 0 37/31/2008
  • Slide 4
  • Overview of Ch. 9 Simple Fixed-Sample-Size Tests Composite Fixed-Sample-Size Tests The -2 log Approximation The Analysis of Variance (ANOVA) Multivariate Methods ANOVA: the Repeated Measures Case Bootstrap Methods: the Two-sample t-test Sequential Analysis 47/31/2008
  • Slide 5
  • Simple Fixed-Sample-Size Tests 57/31/2008
  • Slide 6
  • The Issue In the simplest case, everything is specified Probability distribution of H 0 and H 1 Including all parameters (and K) But: is left unspecified It is desirable to have a procedure that minimizes given a fixed This would maximize the power of the test 1-, the probability of rejecting H 0 when H 1 is true 67/31/2008
  • Slide 7
  • Most Powerful Procedure Neyman-Pearson Lemma States that the likelihood-ratio (LR) test is the most powerful test for a given The LR is defined as: where f 0, f 1 are completely specified density functions for H 0,H 1 X 1, X 2, X n are iid random variables 77/31/2008
  • Slide 8
  • Neyman-Pearson Lemma H 0 is rejected when LR K With a constant K chosen such that: P(LR K when H 0 is true) = Lets look at an example using the Neyman- Pearson Lemma! Then we will prove it. 87/31/2008
  • Slide 9
  • Example Basketball players seem to be taller than average Use this observation to formulate our hypothesis H 1 : Tallness is a factor in the recruitment of KU basketball players The null hypothesis, H 0, could be: No, the players on KUs team are a just average height compared to the population in the U.S. Average height of the team and the population in general is the same 97/31/2008
  • Slide 10
  • Example Setup: Average height of males in the US: 59 Average height of KU players in 2008: 604 Assumption: both populations are normal-distributed centered on their respective averages ( 0 = 69.5 in, 1 = 76.5 in) and = 2 Sample size: 3 Choose : 5% 107/31/2008
  • Slide 11
  • Example The two populations: height (inches) p f0f0 f1f1 117/31/2008
  • Slide 12
  • Example Our test statistic is the Likelihood Ratio, LR Now we need to determine a significance point K at which we can reject H 0, given = 5% P((x) K | H 0 is true) = 0.05, determine K 127/31/2008
  • Slide 13
  • Example So we just need to solve for K and calculate K: How to solve this? Well, we only need one set of values to calculate K, so lets pick two and solve for the third: We get one result: K 3 =71.0803 137/31/2008
  • Slide 14
  • Example Then we can just plug it in to and calculate K: 147/31/2008
  • Slide 15
  • Example With the significance point K = 1.663*10 -7 we can now test our hypothesis based on observations: E.g.: Sasha = 83 in, Darrell = 81 in, Sherron = 71 in 1.446*10 12 > 1.663*10 -7 Therefore, our hypothesis that tallness is a factor in the recruitment of KU basketball players is true. 157/31/2008
  • Slide 16
  • Neyman-Pearson Proof Let A define region in the joint range of X 1, X 2, X n such that LR K. A is the critical region. If A is the only critical region of size we are done Lets assume another critical region of size , defined by B 167/31/2008
  • Slide 17
  • Proof H 0 is rejected if the observed vector (x 1, x 2, , x n ) is in A or in B. Let A and B overlap in region C Power of the test: rejecting H 0 when H 1 is true The Power of this test using A is: 177/31/2008
  • Slide 18
  • Proof Define: = A L(H 1 ) - B L(H 1 ) The power of the test using A minus using B Where A\C is the set of points in A but not in C And B\C contains points in B but not in C 187/31/2008
  • Slide 19
  • Proof So, in A\C we have: While in B\C we have: 197/31/2008 Why?
  • Slide 20
  • Proof Thus Which implies that the power of the test using A is greater than or equal to the power using B. 207/31/2008
  • Slide 21
  • Composite Fixed-Sample-Size Tests 217/31/2008
  • Slide 22
  • Not Identically Distributed In most cases, random variables are not identically distributed, at least not in H 1 This affects the likelihood function, L For example, H 1 in the two-sample t-test is: Where 1 and 2 are different 227/31/2008
  • Slide 23
  • Composite Further, the hypotheses being tested do not specify all parameters They are composite This chapter only outlines aspects of composite test theory relevant to the material in this book. 237/31/2008
  • Slide 24
  • Parameter Spaces The set of values the parameters of interest can take Null hypothesis: parameters in some region Alternate hypothesis: parameters in is usually a subspace of Nested hypothesis case Null hypothesis nested within alternate hypothesis This book focuses on this case if the alternate hypothesis can explain the data significantly better we can reject the null hypothesis 247/31/2008
  • Slide 25
  • Ratio Optimality theory for composite tests suggests this as desirable test statistic: L max (): maximum likelihood when parameters are confined to the region L max (): maximum likelihood when parameters are confined to the region , defined by H 1 H 0 is rejected when is sufficiently small ( Type I error) 257/31/2008
  • Slide 26
  • Example: t-tests The next slides calculate the -ratio for the two sample t-test (with the likelihood) t-tests later generalize to ANOVA and T 2 tests 267/31/2008
  • Slide 27
  • Equal Variance Two-Sided t-test Setup Random variables X 11,,X 1m in group 1 are Normally and Independently Distributed ( 1, 2 ) Random variables X 21,,X 2n in group 2 are NID ( 2, 2 ) X 1i and X 2j are independent for all i and j Null hypothesis H 0 : 1 = 2 (= , unspecified) Alternate hypothesis H 1 : both unspecified 277/31/2008
  • Slide 28
  • Equal Variance Two-Sided t-test Setup (continued) 2 is unknown and unspecified in H 0 and H 1 Is assumed to be the same in both distributions Region is: Region is: 287/31/2008
  • Slide 29
  • Equal Variance Two-Sided t-test Derivation H 0 : writing for the mean, when 1 = 2, the maximum over likelihood is at And the (common) variance 2 is 297/31/2008
  • Slide 30
  • Equal Variance Two-Sided t-test Inserting both into the likelihood function, L 307/31/2008
  • Slide 31
  • Equal Variance Two-Sided t-test Do the same thing for region Which produces this likelihood Function, L 317/31/2008
  • Slide 32
  • Equal Variance Two-Sided t-test The test statistic is then Its the same function, just With different variances 327/31/2008
  • Slide 33
  • Equal Variance Two-Sided t-test We can then use the algebraic identity To show that Where t is (from Ch. 3) 337/31/2008
  • Slide 34
  • Equal Variance Two-Sided t-test t is the observed value of T S is defined in Ch. 3 as We can plot as a function of t: (e.g. m+n=10) t 347/31/2008
  • Slide 35
  • Equal Variance Two-Sided t-test So, by the monotonicity argument, we can use t 2 or |t| instead of as test statistic Small values of correspond to large values of |t| Sufficiently large |t| lead to rejection of H 0 The H 0 distribution of t is known t-distribution with m+n-2 degrees of freedom Significance points are widely available Once has been chosen, values of |t| sufficiently large to reject H 0 can be determined 357/31/2008
  • Slide 36
  • Equal Variance Two-Sided t-test http://www.socr.ucla.edu/Applets.dir/T-table.html 367/31/2008
  • Slide 37
  • Equal Variance One-Sided t-test Similar to Two-Sided t-test case Different region for H 1 : Means 1 and 2 are not simply different, but one is larger than the other 1 2 If then maximum likelihood estimates are the same as for the two-sided case 377/31/2008
  • Slide 38
  • Equal Variance One-Sided t-test If then the unconstrained maximum of the likelihood is outside of The unique maximum is at, implying that the maximum in occurs at a boundary point in At this point estimates of 1 and 2 are equal At this point the likelihood ratio is 1 and H 0 is not rejected Result: H 0 is rejected in favor of H 1 ( 1 2 ) only for sufficiently large positive values of t 387/31/2008
  • Slide 39
  • Example - Revised This scenario fits with our original example: H 1 is that the average height of KU basketball players is bigger than for the general population One-sided test We could assume that we dont know the averages for H 0 and H 1 We actually dont know (I