One-way ANOVA Model Assumptions - homepage.stat.uiowa.edu

32
One-way ANOVA Model Assumptions STAT:5201 Week 4: Lecture 1 1 / 32

Transcript of One-way ANOVA Model Assumptions - homepage.stat.uiowa.edu

Page 1: One-way ANOVA Model Assumptions - homepage.stat.uiowa.edu

One-way ANOVA Model Assumptions

STAT:5201

Week 4: Lecture 1

1 / 32

Page 2: One-way ANOVA Model Assumptions - homepage.stat.uiowa.edu

One-way ANOVA: Model Assumptions

Consider the single factor model: Yij = µ+ αi︸ ︷︷ ︸+εij with εijiid∼ N(0, σ2)

↑ ↑mean structure random

Here, the assumptions are coming from the errors:1 Normally distributed2 Constant variance3 Independent

We will use the estimated errors εij = Yij − Yij or residuals from themodel to check the assumptions.(Though internally studentized residuals or externally studentized residuals

can also be used.)

4 Adequacy of the model is about the mean structure, and here, we arefitting the most complex mean structure for this single factor study,which is a separate mean for each group, so in the full model, thisshouldn’t be a concern.

2 / 32

Page 3: One-way ANOVA Model Assumptions - homepage.stat.uiowa.edu

One-way ANOVA: Model Assumptions

Consider the single factor model: Yij = µ+ αi︸ ︷︷ ︸+εij with εijiid∼ N(0, σ2)

↑ ↑mean structure random

If the model is correct, our inferences are good.

If the assumptions are not true, our inferences may not be valid.

Confidence intervals might not cover at the stated level.p-values are not necessarily valid.Type I error rates could be larger (or smaller) than stated.

Assumptions need to be checked for the validity of the tests.

3 / 32

Page 4: One-way ANOVA Model Assumptions - homepage.stat.uiowa.edu

One-way ANOVA: Model Assumptions

Consider the single factor model: Yij = µ+ αi︸ ︷︷ ︸+εij with εijiid∼ N(0, σ2)

↑ ↑mean structure random

Some procedures work reasonably well even if some of theassumptions are violated (we’ll explore this for the two-sample t-testin homework). This is called robustness of validity.

More mild violations are of course better than more extremeviolations, with respect to validity.

4 / 32

Page 5: One-way ANOVA Model Assumptions - homepage.stat.uiowa.edu

One-way ANOVA: Checking Constant Variance

Checking constant variance

Plot residuals vs. fitted values

If the model is OK for constant variance, then this plot should show arandom scattering of points above and below the reference line at ahorizontal 0, as on the left below. The right one shows nonconstantvariance.

C(.dl~ c~ VfVt/ov~

PLo-/ 01- rM/~ V5, /I/J V~.

7f f'I1 ozid! OK I s.~ 5 /'a.d~ :;ea IIPi I/;:;1jJN'ds ~~ ~ ~Iow· fk- VP[-/-'W r~

;;~ a/~ a . hoY'" 2-crJd 0,

r.e.. ~ "d .,...

••"0, ,

# •

.. '

r: ,. r '

,,

+;.\\--J v~'"y

LlVheJ ~ /J ~ pet 4~VY':'\(=? v"" itlVIee Jl/2 f.v.d.s 6Yl ~ <'aM )

'tL f ..«J ,Cko..Jb.y '.JS_ l'!feol p {".+ ~ ~ 0 !J IWLY (fIA.. s~ ~ cfl--/n1.J,'dYL cd) ~ ~ ~1'7 r fq-!~

l' ~L<' r7 C5~, _ -: ~f~0L ,;; r-: t2zL f/11/jJ/~

??/Yl ~/JL ~J ~~d-YL ~ / c/fu n ~

f}/ c-f wLV &j/~ ~/Jkg- tL Uv, v~ ~

Megaphone pattern violation(variance depends on mean)

5 / 32

Page 6: One-way ANOVA Model Assumptions - homepage.stat.uiowa.edu

One-way ANOVA: Checking Constant Variance

There are some statistical tests that will perform a hypothesis test forthe equality of variances across groups. H0 : σ2i is equal for all i .

Levene’s Test (uses dij = |Yij − Yi·|)Modified Levene’s test (a.k.a. Brown-Forsythe Test, dij = |Yij − Yi·|)Hartley test (need equal sample sizes & normal errors, h =

max(s2i )

min(s2i ))

Example (SAS: levene’s test)

proc glm data=upsit plots=diagnostics;

class agegroup;

model smell = agegroup;

means agegroup / hovtest=levene;

run;

For the other homogeneity of variance (HOV) tests in SAS, justgoogle ‘SAS HOVTEST’. You can use HOVTEST=BF forbrown-Forsythe test.

6 / 32

Page 7: One-way ANOVA Model Assumptions - homepage.stat.uiowa.edu

One-way ANOVA: Checking Constant Variance

Example (SAS: levene’s test)

The SAS System 12:31 Thursday, February 1, 2018 2

The GLM Procedure

Dependent Variable: smell

The SAS System 12:31 Thursday, February 1, 2018 2

The GLM Procedure

Dependent Variable: smell

Source DFSum of

Squares Mean Square F Value Pr > F

Model 4 2.13878141 0.53469535 16.65 <.0001

Error 175 5.61970399 0.03211259

Corrected Total 179 7.75848539

R-Square Coeff Var Root MSE smell Mean

0.275670 14.52664 0.179200 1.233594

Source DF Type I SS Mean Square F Value Pr > F

agegroup 4 2.13878141 0.53469535 16.65 <.0001

Source DF Type III SS Mean Square F Value Pr > F

agegroup 4 2.13878141 0.53469535 16.65 <.0001

0.6

0.8

1.0

1.2

1.4

smel

l

1 2 3 4 5

agegroup

Distribution of smell

0.6

0.8

1.0

1.2

1.4

smel

l

1 2 3 4 5

agegroup

<.0001Prob > F16.65F

Distribution of smell

The SAS System 12:31 Thursday, February 1, 2018 3

The GLM Procedure

The SAS System 12:31 Thursday, February 1, 2018 3

The GLM Procedure

Levene's Test for Homogeneity of smell VarianceANOVAof Squared Deviations from Group Means

Source DFSum of

SquaresMean

Square F Value Pr > F

agegroup 4 0.0799 0.0200 6.35 <.0001

Error 175 0.5503 0.00314

The plot suggests we have nonconstant variance, and the nullhypothesis of H0 : σ1 = σ2 = · · ·σ5 is strongly rejected.

Oehlert has some issues with using statistical tests for nonconstant variance due to

sensitivity to non-normality, but you may still be asked about these tests.7 / 32

Page 8: One-way ANOVA Model Assumptions - homepage.stat.uiowa.edu

One-way ANOVA: Checking Constant Variance

Example (SAS: levene’s test, resids vs.!fitted)

The usual diagnostic plot of residuals vs. fitted values would haverevealed the issue as well (include plot=diagnostics in the procstatement to get the plots).

8 / 32

Page 9: One-way ANOVA Model Assumptions - homepage.stat.uiowa.edu

One-way ANOVA: Model Adequacy

Plot of residuals vs. fitted values can help diagnose model inadequacy.

The residuals vs. fitted plot can also give you some information aboutthe adequacy of the model in a multi-factor ANOVA (we’ll see thislater in multi-factor factorials, plot shown below).

For instance, if you are missing an important interaction term in themean structure, then this plot will often display a curved trend.

9 / 32

Page 10: One-way ANOVA Model Assumptions - homepage.stat.uiowa.edu

One-way ANOVA: Dealing with Nonconstant Variance

Dealing with nonconstant variance

When the variance depends on the mean (like in the megaphonepattern), the usual approach is to apply a transformation to theresponse variable.

These are called variance-stabilizing transformations.

Suppose Var(y) ∝ mean or Var(y) ∝ µ

I want a transformation of y , or function h(y), such thatVar [h(y)]=constant.

Consider a Taylor Series expansion of h around µ

10 / 32

Page 11: One-way ANOVA Model Assumptions - homepage.stat.uiowa.edu

One-way ANOVA: Dealing with Nonconstant Variance

We now have the first-order approximation:

Var [h(y)] ≈ Var [h(µ) + h′(µ)(y − µ)]= [h′(µ)]2Var(y)= c0[h′(µ)]2µ {as Var(y) ∝ µ}

And we want Var [h(y)] to be a constant.

So, set [h′(µ)]2µ equal to a constant and solve for h.

11 / 32

Page 12: One-way ANOVA Model Assumptions - homepage.stat.uiowa.edu

One-way ANOVA: Dealing with Nonconstant Variance

Setting equal to a constant and solving for the unknown h:

[h′(µ)]2µ =constant

⇒ h′(µ) =

√constant

µ

⇒ h(µ) =c1

∫1

µ1/2dµ

⇒ h(µ) =c2√µ

So, if Var(y) ∝ µ, then use a square root transformation to achieveconstant variance.

12 / 32

Page 13: One-way ANOVA Model Assumptions - homepage.stat.uiowa.edu

One-way ANOVA: Dealing with Nonconstant Variance

This is built on the Delta Method.

If Var(y) ∝ mean, use h(y) =√y .

If Var(y) ∝ mean2, use h(y) = ln(y)

Many times it’s hard to tell from the data what the specificrelationship between the variance and the mean is, so a trial-and-errorprocess is applied.

Other possibilities if the spread increases with µ:

h(y) = −1y .

h(y) = log10(y)

Other possibilities if the spread decreases with µ:

h(y) = y2.h(y) = y1.5

13 / 32

Page 14: One-way ANOVA Model Assumptions - homepage.stat.uiowa.edu

One-way ANOVA: Dealing with Nonconstant Variance

The Box-Cox procedure chooses a transformation based on theobserved data. The λ parameter dictates the transformation. Thefollowing form for the transformation is suggested to create the newoutcome variable y (λ):

y (λ) =

{yλ−1λ when λ 6= 0

log(y) when λ = 0

Though λ is continuous, in practice we usually use a convenient λthat is near to the optimal, like 0.5 or 0, etc.

14 / 32

Page 15: One-way ANOVA Model Assumptions - homepage.stat.uiowa.edu

One-way ANOVA: Dealing with Nonconstant Variance

Example (SAS: Box-Cox for 2-factor factorial, perceived as singlefactor or a ‘superfactor’. Weeks has 5 levels, Water has 2 levels.)

proc transreg data=germ;model boxcox(germination)=class(superfactor);

run;

A transformation using λ = 0.25 is suggested, but the convenient λ = 0.5looks to be in the confidence interval (we will check constant varianceafter the transformation).

15 / 32

Page 16: One-way ANOVA Model Assumptions - homepage.stat.uiowa.edu

One-way ANOVA: Dealing with Nonconstant Variance

Example (R: Box-Cox for 2-factor factorial, perceived as singlefactor or a ‘superfactor’. Weeks has 5 levels, Water has 2 levels.)

> library(MASS)

> bc.fit <- boxcox(germination∼as.factor(superfactor))> bc.fit

16 / 32

Page 17: One-way ANOVA Model Assumptions - homepage.stat.uiowa.edu

One-way ANOVA: Dealing with Nonconstant Variance

So, if we know the proportional relationship between the variance andthe mean, then we can analytically find an appropriate transformationto achieve constant variance (or near constant variance).

When we do not know the relationship, then we can apply theBox-Cox transformation to give us a suggestion of an appropriatetransformation.

In practice, if I observe a nonconstant variance that can be correctedthrough transformation (not all of them are), I mostly see varianceincreasing with the mean, and I just try a square-root or logtransformation right away (or log(Y + 1) if there are zeros.).

REMINDER: In a two-sample t-test with nonconstant variance, we dohave an availalbe method called Welch’s Approximate t or orWelch-Satterthwaite t.

17 / 32

Page 18: One-way ANOVA Model Assumptions - homepage.stat.uiowa.edu

One-way ANOVA: Checking Normality

Checking Normality

This is usually done by plotting a normal probability plot or normalQQ-plot.

If the data were generated from a normal distribution, then thenormal probability plot will show the data points falling approximatelyalong the diagonal reference line (this is not a best-fit line, it simplyconnects the 25th and 75th percentile points).

18 / 32

Page 19: One-way ANOVA Model Assumptions - homepage.stat.uiowa.edu

One-way ANOVA: Checking Normality

Chi-squared, right-skewedChi-squared (df=2)

rchisq(1000, 2)

Frequency

0 5 10 15

0100

200

300

-3 -2 -1 0 1 2 3

02

46

810

Normal Q-Q Plot

Theoretical Quantiles

Sam

ple

Qua

ntile

s

Uniform, thin tailsUniform(0,1)

y

Frequency

0.0 0.2 0.4 0.6 0.8 1.0

010

2030

4050

60

-3 -2 -1 0 1 2 3

0.0

0.2

0.4

0.6

0.8

1.0

Normal Q-Q Plot

Theoretical Quantiles

Sam

ple

Qua

ntile

s

19 / 32

Page 20: One-way ANOVA Model Assumptions - homepage.stat.uiowa.edu

One-way ANOVA: Checking Normality

There are a number of statistical tests that test for non-normality:

Anderson-Darling testShapiro-Wilk testMany others

One issue with normality tests is that as your N gets larger, you startto get a lot of power for detecting very small deviations fromnormality. In small samples, you’ll probably never reject.

In practice, I feel like the visual normal probability plot is most useful.But clients will commonly ask how to perform certain tests, such asthese, in software.

20 / 32

Page 21: One-way ANOVA Model Assumptions - homepage.stat.uiowa.edu

One-way ANOVA: Dealing with Non-normality

Dealing with Non-normality

Try a transformation.

If there’s an outlier that a transformation does not fix, do a‘sensitivity analysis’ where you perform the analysis with and withoutthe outlier. These can both be reported. If the important items donot change (e.g. significance) the outlier is perhaps not a big issue.

DO NOT simply remove an outlier because it is unusual!! Not OK.(See link on webpage). And in fact, that data point could be tellingyou something very interesting.

Try a non-parametric test:Randomization test (pretty general)Wicoxon rank-sum test/Mann-Whitney test (for 2-sample t-test)Wilcoxon signed-rank test (for paired t-test)Kruskal-Wallis test (for 1-way ANOVA, extends Mann-Whitney test)Freidman test (for RCBD)

21 / 32

Page 22: One-way ANOVA Model Assumptions - homepage.stat.uiowa.edu

One-way ANOVA: Checking Independence

Checking Independence

Many times, a client brings the data to you and you have to rely ontheir description of the data collection, and that independence holds.

Ideally, it should be part of the design in terms of randomly assigningtreatments to EUs and randomly assigning the order of the runs.

If you happen to know the order in which data were collected, or thetime the observations were collected, it’s a good idea to check forcorrelation in the residuals with respect to order or time (e.g. plotresids vs. time and/or resids vs. order).

22 / 32

Page 23: One-way ANOVA Model Assumptions - homepage.stat.uiowa.edu

One-way ANOVA: Checking Independence

If a pattern appears in the plots, then these items are sources ofvariation that can be added to the model (though it makes the modela bit more complex).

Above we see sequential observations over time, and an observabletrend.

23 / 32

Page 24: One-way ANOVA Model Assumptions - homepage.stat.uiowa.edu

One-way ANOVA: Checking Independence

The Durbin-Watson test statistic can be used to check for timedependence or serial dependence. The residuals ei are used tocalculate DW:

DW =∑n−1

i (ei−ei−1)2∑n

i e2i

Independent data tend to have DW around 2. A positive correlationmakes DW smaller and negative correlation makes it bigger.

If DW gets as low as 1.5 or as high as 2.5, you should start worryingabout time correlation and it’s affect on the inference.

24 / 32

Page 25: One-way ANOVA Model Assumptions - homepage.stat.uiowa.edu

One-way ANOVA: Checking Independence

Example (Checking residuals for time correlation)

The SAS data set diags contains a vector of residuals under the columnname resid. Below, nothing is listed as a predictor in the model statementbecause only an intercept is used.

proc reg data=diags;

model resid= /DW;

run; The SAS System 14:15 Thursday, February 1, 2018 2

The REG ProcedureModel: MODEL1

Dependent Variable: Resid Residual

The SAS System 14:15 Thursday, February 1, 2018 2

The REG ProcedureModel: MODEL1

Dependent Variable: Resid Residual

Durbin-Watson D 1.319

Number of Observations 48

1st Order Autocorrelation 0.334

An approximate 95% CI for ρ is 0.334± 2 ∗ 1√n

or 0.334± 0.289, so there

appears to be correlation in the residuals over time.25 / 32

Page 26: One-way ANOVA Model Assumptions - homepage.stat.uiowa.edu

One-way ANOVA: Dealing with Dependence

Another type of correlation is a spatial correlation, which could bechecked using a variogram.

If there is some kind of non-independence, we should incorporate thisinto our model.

Perhaps there is time-correlation or spatial-correlation, and we havemodels that will incorporate this correlation structure.

If there are repeated measures on a single subject, then this alsorepresents correlation (within the observations on a person), and wecan incorporate that into our model.

If the residuals are non-independent because you were missing animportant factor, then we can include that factor in the model.

26 / 32

Page 27: One-way ANOVA Model Assumptions - homepage.stat.uiowa.edu

Importance of Assumption Violations

Biggest Issue: Non-independence

Standard errors for treatments can be biased, and this can greatlyaffect the type I error rate of our test.

Next Biggest Issue: Nonconstant variance

If you have balanced data, then the affect on p-values is potentiallysmall. For unbalanced data, the error rates can be greatly affected.

Smallest Issue: Non-normality

If you have ‘moderate’ non-normality, the p-values are only slightlyaffected. If it’s ‘very’ non-normal, inference can be affected. Similarly,one very strong outlier can greatly affect the results. Again, this willhave the least impact on error rates in balanced data.

27 / 32

Page 28: One-way ANOVA Model Assumptions - homepage.stat.uiowa.edu

Checking Assumptions: Example

Example (Response time for circuit types)

Returning to our previous 1-way ANOVA example to check assumptions...

Three different types of circuit are investigated for response time in milliseconds. Fifteenare completed in a balanced CRD with the single factor of Type (1,2,3).

Circuit Type Response Time

1 9 12 10 8 152 20 21 23 17 303 6 5 8 16 7

From D.C Montgomery (2005). Design and Analysis of Experiments. Wiley:USA

28 / 32

Page 29: One-way ANOVA Model Assumptions - homepage.stat.uiowa.edu

Checking Assumptions: Example

Example (Response time for circuit types)

Normality looks violated.

29 / 32

Page 30: One-way ANOVA Model Assumptions - homepage.stat.uiowa.edu

Checking Assumptions: Example

Example (Response time for circuit types)

We’ll apply the natural log-transformation and perform the 1-way ANOVAon the transformed response.

30 / 32

Page 31: One-way ANOVA Model Assumptions - homepage.stat.uiowa.edu

Checking Assumptions: Example

Example (Response time for circuit types)

Constant variance seems to be worse here.

We will go back and try a nonparametric test on the original data.

31 / 32

Page 32: One-way ANOVA Model Assumptions - homepage.stat.uiowa.edu

Checking Assumptions: Example

Example (Response time for circuit types)

Perform a 1-way ANOVA using a nonparametric test: Kruskal-Wallis.

Kruskal-Wallis Test

Chi-Square 10.3735

DF 2

Asymptotic Pr > Chi-Square 0.0056

Exact Pr >= Chi-Square 0.0005

⇒ Reject H0 : αi = 0 for all i , where HA : at least one group different32 / 32