Confidence intervals and hypothesis testing Petter Mostad 2005.10.03.

of 29/29
Confidence intervals and hypothesis testing Petter Mostad 2005.10.03
  • date post

  • Category


  • view

  • download


Embed Size (px)

Transcript of Confidence intervals and hypothesis testing Petter Mostad 2005.10.03.

  • Confidence intervals and hypothesis testingPetter Mostad2005.10.03

  • Confidence intervals (repetition)Assume and 2 are some real numbers, and assume the data X1,X2,,Xn are a random sample from N(,2). Then

    thus so and we say that is a confidence interval for with 95% confidence, based on the statistic

  • Confidence intervals, general ideaWe have a model with an unknown parameterWe find a statistic (function of the sample) with a known distribution, depending only on the unknown parameterThis distribution is used to construct an interval with the following property: If you repeat many times selecting a parameter and simulating the statistic, then about (say) 95% of the time, the confidence interval will contain the parameter

  • Hypothesis testingSelecting the most plausible model for the data, among those suggestedExample: Assume X1,X2,,Xn is a random sample from N(,2), where 2 is known, but is not; we want to select fitting the data. One possibility is to look at the probability of observing the data given different values for . (We will return to this)Another is to do a hypothesis test

  • Example We select two alternative hypotheses: H0: H1: Use the value of to test H0 versus H1: If is far from , it will indicate H1. Under H0, we know that

    Reject H0 if is outside

  • General outline for hypothesis testingThe possible hypotheses are divided into H0, the null hypothesis, and H1, the alternative hypothesisA hypothesis can beSimple, so that it is possible to compute the probability of data (e.g., )Composite, i.e., a collection of simple hypotheses (e.g., )

  • General outline (cont.)A test statistic is selected. It must: Have a higher probability for extreme values under H1 than under H0Have a known distribution under H0 (when simple)If the value of the test statistic is too extreme, then H0 is rejected. The probability, under H0, of observing the given data or something more extreme is called the p-value. Thus we reject H0 if the p-value is small.The value at which we reject H0 is called the significance level.

  • Note: There is an asymmetry between H0 and H1: In fact, if the data is inconclusive, we end up not rejecting H0. If H0 is true the probability to reject H0 is (say) 5%. That DOES NOT MEAN we are 95% certain that H0 is true! How much evidence we have for choosing H1 over H0 depends entirely on how much more probable rejection is if H1 is true.

  • Errors of types I and IIThe above can be seen as a decision rule for H0 or H1. For any such rule we can compute (if both H0 and H1 are simple hypotheses): Accept H0Reject H0H0 trueH1 trueTYPE II error TYPE I errorSignificance1 - power

    P(accept | H0)P(accept | H1)P(reject | H0)P(reject | H1)

  • Significance and powerIf H0 is composite, we compute the significance from the simple hypothesis that gives the largest probability of rejecting H0. If H1 is composite, we compute a power value for each simple hypothesis. Thus we get a power function.

  • Example 1: Normal distribution with unknown varianceAssume



    So a confidence interval for , with significance is given by

  • Example 1 (Hypothesis testing)Hypotheses:

    Test statistic under H0

    Reject H0 if or if

    Alternatively, the p-value for the test can be computed (if ) as the such that

  • Example 1 (cont.)Hypotheses:

    Test statistic assuming

    Reject H0 if

    Alternatively, the p-value for the test can be computed as the such that

  • Example 1 (cont.)Assume that you want to analyze as above the data in some column of an SPSS table. Use Analyze => Compare means => One-sample T TestYou get as output a confidence interval, and a test as the one described above. You may adjust the confidence level using Options

  • Example 2: Differences between meansAssume and

    We would like to study the difference Four different cases:Matched pairsKnown population variancesUnknown but equal population variancesUnknown and possibly different pop. variances

  • Known population variancesWe get

    Confidence interval for

  • Unknown but equal population variancesWe get


    Confidence interval for

  • Hypothesis testing: Unknown but equal population variancesHypotheses:

    Test statistic:

    Reject H0 if or if T test with equal variances

  • Unknown and possibly unequal population variancesWe get


    Conf. interval for

  • Hypothesis test: Unknown and possibly unequal pop. variancesHypotheses:

    Test statistic

    Reject H0 if or if T test with unequal variances

  • Practical examples:The lengths of children in a class are measured at age 8 and at age 10. Use the data to find an estimate, with confidence limits, on how much children grow between these ages. You want to determine whether a costly operation is generally done more cheaply in France than in Norway. Your data is the actual costs of 10 such operations in Norway and 20 in France.

  • Example 3: Population proportionsAssume , so that is a frequency.Then



    Confidence interval for (approximately, for large n)(approximately, for large n)

  • Example 3 (Hypothesis testing)Hypotheses:

    Test statistic under H0, for large n

    Reject H0 if or if

  • Example 4: Differences between population proportionsAssume and , so that and are frequencies


    Confidence interval for (approximately)

  • Example 4 (Hypothesis testing)Hypotheses:

    Test statistic


    Reject H0 if

  • Example 5: The variance of a normal distributionAssumeThen


    Confidence interval for

  • Example 6: Comparing variances for normal distributionsAssumeWe get

    Fnx-1,ny-1 is an F distribution with nx-1 and ny-1 degrees of freedomWe can use this exactly as before to obtain a confidence interval for and for testing for example if Note: The assumption of normality is crucial!

  • Sample size computationsFor a sample from a normal population with known variance, the size of the conficence interval for the mean depends only on the sample size. So we can compute the necessary sample size to match a required accuracyNote: If the variance is unknown, it must somehow be estimated on beforehand to do the computationWorks also for population proportion estimation, giving an inequality for the required sample size

  • Power computationsIf you reject H0, you know very little about the evidence for H1 versus H0 unless you study the power of the test. The power is 1 minus the probability of rejecting H0 given that a hypothesis in H1 is true. Thus it is a function of the possible hypotheses in H1. We would like our tests to have as high power as possible.

    Note: As we can compute the interval if we have sigma, the above can be used in practice in cases where sigma is known, and we have a sample. Note: The interval is connected not only with the unknown parameter, but also with the statistic used: Different statistics can give different intervals. Contrast this with credibility intervals, where you actually talk about the knowledge you have about the parameter. Notice the OPPOSITE ways of using the inequalities for CONFIDENCE INTERVALS and HYPOTHESIS TESTING. Example: One could in principle just simulate a standard normal random variable, independently of data, and reject H0 based on extreme values of that. Matched pairs are done directlyThe hypothesis testing can be done correspondingly