Hypothesis TestingA procedure for determining which of

two (or more) mutually exclusive statements is more likely true

We classify hypothesis tests in terms of – Parametric vs. Non-parametric– Directional vs. Non-directional

Hypothesis TestsParametric Tests - tests about specific

population parameters (μ, σ2, etc.)– Is μ1 different from a predetermined

value?– Is μ1 different from μ2 ?

Non-Parametric Tests - tests about the shape of the population (medians?)– Is this population different from another

population?

Hypothesis TestsNon-Directional Hypothesis

– Is μ1 different from μ2 ?

– Is the distribution of scores in group 1 different than those in group 2?

Directional Hypotheses– Is μ1 greater than μ2 ?

– Is μ1 less than μ2 ?

– Is the distribution of scores in group 1 to the right (greater than) of those in group 2?

The Six-Steps ofHypothesis Testing

1. State and Check Assumptions2. Generate Null and Alternative

Hypotheses3. Chose the Sampling Distribution of the

Test Statistic4. Set Significance Level5. Compute the Test Statistic6. Draw Conclusions

1. State and Check AssumptionsThere are three requirements for

hypothesis testing to work– Assumptions about the population– Assumptions about the sample

Assumptions about the Population“Assumption of Normality” - the

population is normally distributed or the sample size is sufficiently large so that the CLT comes into play and

Assumptions about VarianceDepending of the type of test, some

important features of variance may come into play– Is σ or σ2 known?– “Homogeneity of Variance” - the variance

of the populations being compared have equal variance

Assumptions about the SampleThe sample has been obtained using

independent random sampling

What if the assumptions can’t be met?“Violation” - when assumptions are not

met– Violation of Normality = “Non-normal”– Violation of Homogeneity of Variance =

“Heterogeneous Variance”“Robust” - a test’s ability to “deal with”

violations – “a t-test is robust to violations of

normality”

BUT, ...

No tests are robust to violations of the random sampling assumption.

If you do not have a random sample, probability theory will not work and therefore inferential statistical techniques will fail.

2. Generate Null and Alternative HypothesesThese two hypotheses, designated HO

and HA (or H1), are mutually exclusive– mutually exclusive - the don’t overlap

Properties of HO

Specifies no difference or no change from a standard or theoretical value

Always specifies something about a particular population parameter

Used in constructing a sampling distribution– For the subsequent quantitative work, the

null hypothesis is assumed to be true

Properties of HA (or H1)About the same aspect of the

population as HO

Usually stated in general termsMutually exclusive - no overlap with HO

Used in making a decisionCan be directional or non-directional

Directional vs. Non-directional HAsNon-directional HA - usually stated as

“does not equal” or “is different than”Directional HA - stated as “greater

than” or “less than”– note that a non-directional hypothesis is

equal to the two directional hypotheses “greater than” or “less than”

3. Chose the Sampling DistributionDepending on the type of data,

assumptions, and hypotheses certain distributions of the test statistic require selection

The second half of this course will be devoted to making the “best” decision about which test statistic to choose (z, t, F, etc.)

4. Set Significance LevelHypothesis testing is sometimes called

“significance testing”The significance level is the basis for

making our decision – “rejection region” - the value specified by

the significance level of the test– “critical value” - the value of the test

statistic specified by the significance level that begins the rejection region

Significance LevelThe probability value associated with

the decision rule is called the significance level of the test

Significance level is represented by the Greek letter alpha (α)

The actual value of α is up to you

What is the significance level? Hypothesis testing entails determining which

of two hypotheses (HO and H1) is more likely correct.

But “more likely” is a subjective evaluation on your part.

If you were to obtain a statistic that was unlikely if HO were assume to be true, would you be willing to accept the H1?

How “unlikely” does it need to be for you to be convinced?

Typical Significance level

A Typical significance level : α = .05We are convinced that results are

considered as significant different from the HO when they are in the most extreme 5% (a proportion of .05) of all possible outcomes specified in HO

Using αWith our significance level, we

determine a decision rule– Critical Value – the sufficiently extreme

value of the statistic such that if our statistic is more extreme, we reject the HO

– p-value – if the probability of our statistic, assuming HO is true, is less than α, we reject the HO

5. Compute the Test StatisticFrom the random sample obtained, the

test statistic is computed using various formulae– z-statistic– binomial probability– t-statistic– F-statistic– etc.

6. Draw ConclusionsBased on the test statistic just

computed, we do one of two things:– Reject the null hypothesis and accept the

alternative hypothesis, or– Do not reject the null hypothesis and do

not accept the alternative hypothesisNote that we NEVER accept the null

hypothesis, we only fail to reject it!

More on DecisionsWhen completed, a significance test

tells us the probability of obtaining our results when the null hypothesis is true

p(Results|Ho is True) If that probability is small, smaller than

our significance level (α), it is likely that Ho is not true and we reject it