One-way ANOVA Model Assumptions -

of 32 /32
One-way ANOVA Model Assumptions STAT:5201 Week 4: Lecture 1 1 / 32

Embed Size (px)

Transcript of One-way ANOVA Model Assumptions -

One-way ANOVA Model AssumptionsOne-way ANOVA: Model Assumptions
Consider the single factor model: Yij = µ+ αi +εij with εij iid∼ N(0, σ2)
↑ ↑ mean structure random
Here, the assumptions are coming from the errors: 1 Normally distributed 2 Constant variance 3 Independent
We will use the estimated errors εij = Yij − Yij or residuals from the model to check the assumptions. (Though internally studentized residuals or externally studentized residuals
can also be used.)
4 Adequacy of the model is about the mean structure, and here, we are fitting the most complex mean structure for this single factor study, which is a separate mean for each group, so in the full model, this shouldn’t be a concern.
2 / 32
One-way ANOVA: Model Assumptions
Consider the single factor model: Yij = µ+ αi +εij with εij iid∼ N(0, σ2)
↑ ↑ mean structure random
If the model is correct, our inferences are good.
If the assumptions are not true, our inferences may not be valid.
Confidence intervals might not cover at the stated level. p-values are not necessarily valid. Type I error rates could be larger (or smaller) than stated.
Assumptions need to be checked for the validity of the tests.
3 / 32
One-way ANOVA: Model Assumptions
Consider the single factor model: Yij = µ+ αi +εij with εij iid∼ N(0, σ2)
↑ ↑ mean structure random
Some procedures work reasonably well even if some of the assumptions are violated (we’ll explore this for the two-sample t-test in homework). This is called robustness of validity.
More mild violations are of course better than more extreme violations, with respect to validity.
4 / 32
Checking constant variance
Plot residuals vs. fitted values
If the model is OK for constant variance, then this plot should show a random scattering of points above and below the reference line at a horizontal 0, as on the left below. The right one shows nonconstant variance.
C(.dl~ c~ VfVt/ov~
PLo-/ 01- rM/~ V5, /I/J V~.
7f f'I1 ozid! OK I s.~ 5 /'a.d~ :;ea IIPi I/;:; 1jJN'ds ~~ ~ ~Iow· fk- VP[-/-'W r~
;;~ a/~ a . hoY'" 2-crJd 0,
r. e.. ~ "d .,...
+;.\\--J v~ '"y
LlVheJ ~ /J ~ pet 4~VY':'\ (=? v"" itlVIee Jl/2 f.v.d.s 6Yl ~ <'aM )
'tL f ..«J ,Cko..Jb.y '.JS_ l'!feol p {".+ ~ ~ 0 !J IWL Y (fIA.. s~ ~ cfl--/n1.J,'dYL cd) ~ ~ ~1'7 r fq-!~
l' ~L<' r7 C5~, _ -: ~f~0L ,;; r-: t2zL f/11/jJ/~
??/Yl ~/JL ~J ~~d-YL ~ / c/fu n ~
f}/ c-f wLV &j/~ ~/Jkg- tL Uv, v~ ~
Megaphone pattern violation (variance depends on mean)
5 / 32
One-way ANOVA: Checking Constant Variance
There are some statistical tests that will perform a hypothesis test for the equality of variances across groups. H0 : σ2i is equal for all i .
Levene’s Test (uses dij = |Yij − Yi·|) Modified Levene’s test (a.k.a. Brown-Forsythe Test, dij = |Yij − Yi·|) Hartley test (need equal sample sizes & normal errors, h =
max(s2i )
min(s2i ) )
proc glm data=upsit plots=diagnostics;
class agegroup;
For the other homogeneity of variance (HOV) tests in SAS, just google ‘SAS HOVTEST’. You can use HOVTEST=BF for brown-Forsythe test.
6 / 32
The SAS System 12:31 Thursday, February 1, 2018 2
The GLM Procedure
Dependent Variable: smell
The GLM Procedure
Dependent Variable: smell
Model 4 2.13878141 0.53469535 16.65 <.0001
Error 175 5.61970399 0.03211259
Corrected Total 179 7.75848539
0.275670 14.52664 0.179200 1.233594
Source DF Type I SS Mean Square F Value Pr > F
agegroup 4 2.13878141 0.53469535 16.65 <.0001
Source DF Type III SS Mean Square F Value Pr > F
agegroup 4 2.13878141 0.53469535 16.65 <.0001
The GLM Procedure
The GLM Procedure
Levene's Test for Homogeneity of smell Variance ANOVAof Squared Deviations from Group Means
Source DF Sum of
agegroup 4 0.0799 0.0200 6.35 <.0001
Error 175 0.5503 0.00314
The plot suggests we have nonconstant variance, and the null hypothesis of H0 : σ1 = σ2 = · · ·σ5 is strongly rejected.
Oehlert has some issues with using statistical tests for nonconstant variance due to
sensitivity to non-normality, but you may still be asked about these tests. 7 / 32
One-way ANOVA: Checking Constant Variance
Example (SAS: levene’s test, resids vs.!fitted)
The usual diagnostic plot of residuals vs. fitted values would have revealed the issue as well (include plot=diagnostics in the proc statement to get the plots).
8 / 32
One-way ANOVA: Model Adequacy
Plot of residuals vs. fitted values can help diagnose model inadequacy.
The residuals vs. fitted plot can also give you some information about the adequacy of the model in a multi-factor ANOVA (we’ll see this later in multi-factor factorials, plot shown below).
For instance, if you are missing an important interaction term in the mean structure, then this plot will often display a curved trend.
9 / 32
Dealing with nonconstant variance
When the variance depends on the mean (like in the megaphone pattern), the usual approach is to apply a transformation to the response variable.
These are called variance-stabilizing transformations.
Suppose Var(y) ∝ mean or Var(y) ∝ µ
I want a transformation of y , or function h(y), such that Var [h(y)]=constant.
Consider a Taylor Series expansion of h around µ
10 / 32
One-way ANOVA: Dealing with Nonconstant Variance
We now have the first-order approximation:
Var [h(y)] ≈ Var [h(µ) + h′(µ)(y − µ)] = [h′(µ)]2Var(y) = c0[h′(µ)]2µ {as Var(y) ∝ µ}
And we want Var [h(y)] to be a constant.
So, set [h′(µ)]2µ equal to a constant and solve for h.
11 / 32
One-way ANOVA: Dealing with Nonconstant Variance
Setting equal to a constant and solving for the unknown h:
[h′(µ)]2µ =constant
⇒ h(µ) =c2 √ µ
So, if Var(y) ∝ µ, then use a square root transformation to achieve constant variance.
12 / 32
This is built on the Delta Method.
If Var(y) ∝ mean, use h(y) = √ y .
If Var(y) ∝ mean2, use h(y) = ln(y)
Many times it’s hard to tell from the data what the specific relationship between the variance and the mean is, so a trial-and-error process is applied.
Other possibilities if the spread increases with µ:
h(y) = −1 y .
h(y) = y2. h(y) = y1.5
One-way ANOVA: Dealing with Nonconstant Variance
The Box-Cox procedure chooses a transformation based on the observed data. The λ parameter dictates the transformation. The following form for the transformation is suggested to create the new outcome variable y (λ):
y (λ) =
log(y) when λ = 0
Though λ is continuous, in practice we usually use a convenient λ that is near to the optimal, like 0.5 or 0, etc.
14 / 32
One-way ANOVA: Dealing with Nonconstant Variance
Example (SAS: Box-Cox for 2-factor factorial, perceived as single factor or a ‘superfactor’. Weeks has 5 levels, Water has 2 levels.)
proc transreg data=germ; model boxcox(germination)=class(superfactor);
A transformation using λ = 0.25 is suggested, but the convenient λ = 0.5 looks to be in the confidence interval (we will check constant variance after the transformation).
15 / 32
One-way ANOVA: Dealing with Nonconstant Variance
Example (R: Box-Cox for 2-factor factorial, perceived as single factor or a ‘superfactor’. Weeks has 5 levels, Water has 2 levels.)
> library(MASS)
One-way ANOVA: Dealing with Nonconstant Variance
So, if we know the proportional relationship between the variance and the mean, then we can analytically find an appropriate transformation to achieve constant variance (or near constant variance).
When we do not know the relationship, then we can apply the Box-Cox transformation to give us a suggestion of an appropriate transformation.
In practice, if I observe a nonconstant variance that can be corrected through transformation (not all of them are), I mostly see variance increasing with the mean, and I just try a square-root or log transformation right away (or log(Y + 1) if there are zeros.).
REMINDER: In a two-sample t-test with nonconstant variance, we do have an availalbe method called Welch’s Approximate t or or Welch-Satterthwaite t.
17 / 32
Checking Normality
This is usually done by plotting a normal probability plot or normal QQ-plot.
If the data were generated from a normal distribution, then the normal probability plot will show the data points falling approximately along the diagonal reference line (this is not a best-fit line, it simply connects the 25th and 75th percentile points).
18 / 32
rchisq(1000, 2)
0 5 10 15
0 2
4 6
8 10
0.0 0.2 0.4 0.6 0.8 1.0
0 10
20 30
40 50
0. 0
0. 2
0. 4
0. 6
0. 8
1. 0
One-way ANOVA: Checking Normality
There are a number of statistical tests that test for non-normality:
Anderson-Darling test Shapiro-Wilk test Many others
One issue with normality tests is that as your N gets larger, you start to get a lot of power for detecting very small deviations from normality. In small samples, you’ll probably never reject.
In practice, I feel like the visual normal probability plot is most useful. But clients will commonly ask how to perform certain tests, such as these, in software.
20 / 32
Dealing with Non-normality
Try a transformation.
If there’s an outlier that a transformation does not fix, do a ‘sensitivity analysis’ where you perform the analysis with and without the outlier. These can both be reported. If the important items do not change (e.g. significance) the outlier is perhaps not a big issue.
DO NOT simply remove an outlier because it is unusual!! Not OK. (See link on webpage). And in fact, that data point could be telling you something very interesting.
Try a non-parametric test: Randomization test (pretty general) Wicoxon rank-sum test/Mann-Whitney test (for 2-sample t-test) Wilcoxon signed-rank test (for paired t-test) Kruskal-Wallis test (for 1-way ANOVA, extends Mann-Whitney test) Freidman test (for RCBD)
21 / 32
Checking Independence
Many times, a client brings the data to you and you have to rely on their description of the data collection, and that independence holds.
Ideally, it should be part of the design in terms of randomly assigning treatments to EUs and randomly assigning the order of the runs.
If you happen to know the order in which data were collected, or the time the observations were collected, it’s a good idea to check for correlation in the residuals with respect to order or time (e.g. plot resids vs. time and/or resids vs. order).
22 / 32
One-way ANOVA: Checking Independence
If a pattern appears in the plots, then these items are sources of variation that can be added to the model (though it makes the model a bit more complex).
Above we see sequential observations over time, and an observable trend.
23 / 32
One-way ANOVA: Checking Independence
The Durbin-Watson test statistic can be used to check for time dependence or serial dependence. The residuals ei are used to calculate DW:
DW = ∑n−1
i e 2 i
Independent data tend to have DW around 2. A positive correlation makes DW smaller and negative correlation makes it bigger.
If DW gets as low as 1.5 or as high as 2.5, you should start worrying about time correlation and it’s affect on the inference.
24 / 32
Example (Checking residuals for time correlation)
The SAS data set diags contains a vector of residuals under the column name resid. Below, nothing is listed as a predictor in the model statement because only an intercept is used.
proc reg data=diags;
run; The SAS System 14:15 Thursday, February 1, 2018 2
The REG Procedure Model: MODEL1
Dependent Variable: Resid Residual
The REG Procedure Model: MODEL1
Dependent Variable: Resid Residual
Number of Observations 48
1st Order Autocorrelation 0.334
An approximate 95% CI for ρ is 0.334± 2 ∗ 1√ n
or 0.334± 0.289, so there
appears to be correlation in the residuals over time. 25 / 32
One-way ANOVA: Dealing with Dependence
Another type of correlation is a spatial correlation, which could be checked using a variogram.
If there is some kind of non-independence, we should incorporate this into our model.
Perhaps there is time-correlation or spatial-correlation, and we have models that will incorporate this correlation structure.
If there are repeated measures on a single subject, then this also represents correlation (within the observations on a person), and we can incorporate that into our model.
If the residuals are non-independent because you were missing an important factor, then we can include that factor in the model.
26 / 32
Biggest Issue: Non-independence
Standard errors for treatments can be biased, and this can greatly affect the type I error rate of our test.
Next Biggest Issue: Nonconstant variance
If you have balanced data, then the affect on p-values is potentially small. For unbalanced data, the error rates can be greatly affected.
Smallest Issue: Non-normality
If you have ‘moderate’ non-normality, the p-values are only slightly affected. If it’s ‘very’ non-normal, inference can be affected. Similarly, one very strong outlier can greatly affect the results. Again, this will have the least impact on error rates in balanced data.
27 / 32
Returning to our previous 1-way ANOVA example to check assumptions...
Three different types of circuit are investigated for response time in milliseconds. Fifteen are completed in a balanced CRD with the single factor of Type (1,2,3).
Circuit Type Response Time
1 9 12 10 8 15 2 20 21 23 17 30 3 6 5 8 16 7
From D.C Montgomery (2005). Design and Analysis of Experiments. Wiley:USA
28 / 32
Normality looks violated.
Example (Response time for circuit types)
We’ll apply the natural log-transformation and perform the 1-way ANOVA on the transformed response.
30 / 32
Constant variance seems to be worse here.
We will go back and try a nonparametric test on the original data.
31 / 32
Perform a 1-way ANOVA using a nonparametric test: Kruskal-Wallis.
Kruskal-Wallis Test
Chi-Square 10.3735
DF 2
Asymptotic Pr > Chi-Square 0.0056
Exact Pr >= Chi-Square 0.0005
⇒ Reject H0 : αi = 0 for all i , where HA : at least one group different 32 / 32
First section