HYPOTHESIS TESTING - ™ƒ„ƒµ»¯´± „…...

download HYPOTHESIS TESTING - ™ƒ„ƒµ»¯´± „… ¤¼­± ... fouskakis/SS/hypothesis Testing Was it due to chance, or something else? Decide

of 24

  • date post

  • Category


  • view

  • download


Embed Size (px)

Transcript of HYPOTHESIS TESTING - ™ƒ„ƒµ»¯´± „…...

  • AMS 5AMS 5


  • Hypothesis TestingHypothesis Testing

    Was it due to chance, or something else?Was it due to chance, or something else?

    Decide between two Decide between two hypotheseshypotheses that are that are mutually exclusive on the basis of mutually exclusive on the basis of evidence from observations.evidence from observations.

  • Test of SignificanceTest of Significance

    A certain brand of tobacco has an average nicotine content of2.5 milligrams, while another one has an average of only 1.5milligrams. A cigarette manufacturer receives an unlabeledshipment of tobacco and needs to determine the nicotinecontent using a sample of the tobacco.

    A certain type of vaccine is known to be only 25% effectiveover a period of 2 years. A new type of vaccine is being testedon 2000 people chosen at random. We want to test if this newvaccine is more effective than the original one.

  • Test of SignificanceTest of Significance A machine for filling bottles of soda has to put 333 ml ofliquid in each bottle. If the average amount is too low or toohigh with respect to the expected content then the machine isconsidered to be out of control. The machine is regularlyinspected to check whether it is out of control by taking asample of bottles. A balance designed for precision weighting in a lab has to bekept very well calibrated. To test the calibration of the balanceseveral measurements of the weight of an object are done. Ifthey differ too much then the balance has to be re-calibrated.

    In all these examples a decision has to be taken based on the numbers in a sample. These numbers are subject to uncertainty and we have to decide if the differences that we observe are only due to chance or not.

  • Null and Alternative HypothesesNull and Alternative HypothesesA bill is proposed to simplify the tax code. The proposer claimsthat the bill is `revenue-neutral', that is, it will not lower taxrevenues. A simulation is run using 100 tax returns chosen atrandom and the differences between the tax paid using the old rules and those that would be paid using the new rules are recorded. The average difference comes up to be -$219 with a standard deviation of $725. Can we claim that the new rules arerevenue-neutral?We can put the problem in these terms: there are two hypotheses:


    ALTERNATIVE HYPOTHESIS, H1Under the null hypothesis there is no difference in revenue and the fact that the observed value is not 0 is totally due to chance. Under the alternative hypothesis the difference is real.

  • Null and Alternative HypothesesNull and Alternative HypothesesFor the examples that we considered at the beginning we have that:

    Cigarette: H0: the mean nicotine content is 1.5. H1: themean nicotine content is 2.5

    Vaccine: H0: the proportion is 25%. H1: the proportion ishigher than 25%.

    Soda Bottles: H0: the average amount of liquid is 333 ml.H1: the average amount of liquid is not equal to 333 ml.

    Balance: H0: the device is calibrated. H1: the device is notproperly calibrated.

  • Test StatisticsTest StatisticsHow do we test the null hypothesis against the alternative?Back to the tax example. Suppose the null is true. Then thedifference should be $0. How `large' is -$219 with respect to $0? To answer this question we convert to standard units. Given thatthe sample was of size 100 the SE is approximately $72:

    $219 $03


    so the difference between the value under the null and the observed value is -3 standard units.The probability of the interval to the left of -3 is about 0.001, thatis, one chance in 1,000. So, under the null hypothesis, $219 is avery unlikely value.

  • Test StatisticsTest Statistics

    In general we are calculating a test statistics given by

    observed expectedz



    which is referred to as the z-test.Once the z-test is calculated we have to decide whether its value is `large' or is `small'. We observe the probability of the left tail of the normal curve, below the z-test. If this probability is small then the value of the z-test is far from the center of the distribution. This probability is called an observed significance level

  • Test StatisticsTest Statistics

    The smaller the P-value, the stronger the evidence against the null, but

  • Making a test of significanceMaking a test of significanceTo make a test of significance you need to:

    set up the null hypothesis pick a test statistics to measure the difference between the data and what is expected under the null hypothesis compute the test statistics and the corresponding observed significance level.

    A small observed significance level implies that the data are far from the values expected from the model under the null hypothesis.What is a small observed significance level?This is somewhat arbitrary, but it is usually considered that if P isless than 5% the results are significant. If P is less than 1% theresults are highly significant.

  • ExamplesExamples1. A random sample of 85 8th graders has a mean score of 265 with an SD of 55 on a national math test. A State Administrator claims that the mean score of 8th graders on the examinations is above 260. Is there enough evidence to support the administrator's claim? The hypotheses are:

    H0: mean 260 vs H1: mean > 260

    The test statistics is obtained by changing to standard units:

    265 2600.838

    55 / 85


    The probability that a standard normal is above 0.838 is about 21%. This is a rather large P-value, so there does not seem to be enough evidence to reject H0.

  • ExamplesExamples2. A light bulb manufacturer guarantees that the mean life of thebulbs is at least 750 hours. A random sample of 36 light bulbshas a mean of 725 hours and a standard deviation of 60 hours.Is there enough evidence to reject the manufacturer's claim?The hypotheses are

    H0: mean 750 vs H1: mean < 750

    The test statistics is obtained by changing to standard units:

    725 7502.05

    60 / 36


    The probability that a standard normal is below -2.05 is about2%. There is some evidence to reject the manufacturer's claim.

  • Binary boxesBinary boxesConsider again the problem of testing the new vaccine. This is abinary model since we can classify the population in two groups:the group of people for which the vaccine was effective and thatfor which the vaccine was not.Under the null hypothesis the box model that generates the sample consists of the box

    since there is 25% chance that the vaccine is effective.Suppose that the number of people in the sample (of 2000 people)for which the vaccine was effective is 534. According to the null hypothesis the expected number would be 500. Is the 34 people difference large enough to reject the null hypothesis and claim that the new vaccine is more effective?

  • Binary boxesBinary boxes

    SD 0.25 0.75 0.43=

    SE 2000 0.43 19.23= =

    We need to calculate the standard units of the difference between 550 and 534. Under the null hypothesis the SD of the box is

    so the standard error is . Then:

    534 500z 1.1768


    = =

    The observed significance level is given by the area under normalcurve corresponding to interval above 1.768. This is around 4%,which is small enough to conclude that the difference is statistically significant. So there is evidence to support the claim that the new vaccine is more effective than the standard.

  • The tThe t--testtestThe examples that we have seen so far rely on the fact that the sample size is large. So, even when the SD of the box is unknown, we can still use the normal curve to obtain the observed significance level of the test.This is not the case when the sample size is small. In this case we need a modification of the z-test due to `Student', a pen name for a statistician called Gosset.Consider the following example. The following five measures of the concentration of Carbon monoxide (CO) are taken from a gassample where the concentration is precisely controlled to be 70 parts per million (ppm). Five measurements are taken to check the calibration of an instrument

    78 83 68 72 88

  • The tThe t--testtest

    7.22 / 5 3.23.

    The null hypothesis is that the device is calibrated and so the average of the measurements is 70 ppm. The average of the sample is 77.8 ppm , the SD is 7.22 ppm and thus the SE of the average is The z-test can be obtained as

    77.8 70z 2.4



    To determine the observed significance level we calculate the areato the right of 2.4 under the normal curve. This is less than 1%,which looks like strong evidence against the null hypothesis.Unfortunately we have to remember that the SD that we havecalculated is NOT the SD of the box. It is the SD of the sample,whose size is fairly small, and thus the approximation is not veryprecise. We correct the procedure with the following steps.

  • The tThe t--testtestStep 1: Consider a different estimate of the SD

    In our previous example we get so the SE of the average becomes as opposed to 3.23, reflecting a higher level of uncertainty.

    Then the test statistics becomes


    number of measurementsSD SD

    number of measurements 1

    Notice that SD SD.



    5SD 7.22 8.07



    8.07 / 5 3.61,=

    77.8 70t 2.2



  • The tThe t--testtestStep 2: To find the observed significance level we can not use thenormal curve any more. We need to use a Student's t curve. Thiscurve depends on the