F tests and the Extra Sum of Squares
Example:
Y = plaster hardness
s = sand content
f = fibre content
Model:
Yi = β0 + β1si + β2s2i + β3fi + β4f
2i + β5si fi + ǫi
In matrix form: Y1...
Yn
=
1 s1 s21 f1 f 2
1 s1f1...
......
......
...1 sn s2
n fn f 2n snfn
β0
...β5
Richard Lockhart STAT 350: F tests
Hypotheses to test
Questions:
◮ Do we need the S ∗ F term?
◮ Do we need the F or F 2 terms?
◮ Do we need the S or S2 terms?
To answer these questions we test
◮ Ho : β5 = 0
◮ Ho : β3 = β4 = 0 (or perhaps Ho : β3 = β4 = β5 = 0)
◮ Ho : β1 = β2 = 0
Richard Lockhart STAT 350: F tests
Technique: we fit a sequence of models:
(a) Original “full” model.
(b) The model with no interactions:
µi = β0 + β1si + β2s2i + β3fi + β4f
2i
(c) The Sand only model:
µi = β0 + β1si + β2s2i
(d) The Fibre only model:
µi = β0 + β3fi + β4f2i
(e) The “Empty” model:µi = β0
Richard Lockhart STAT 350: F tests
Each model has design matrix which is submatrix of full designmatrix:
Y = [1|X1|X2|X3]
β0
β1
β2
β3
β4
β5
+ ǫ
The design matrices for the models a, b, c, d and e are given by
Xa = X
Xb = [1|X1|X2]
Xc = [1|X1]
Xd = [1|X2]
Xe = [1]
Note that 1 is a column vector of n 1s.
Richard Lockhart STAT 350: F tests
F tests
◮ Can compare two models easily if one is a special case of theother.
◮ Example: design matrix of first model is submatrix of secondobtained by selecting subcolumns.
◮ Model (b) is a special case of (a), model (c) is a special caseof (a) or (b) but models (c) and (d) are not comparable.
Richard Lockhart STAT 350: F tests
Comparing two models: General Theory
◮ Consider matrix X partitioned into pieces X1 and X2.
X = [X1|X2]
◮ The Full Model is
Y = Xβ + ǫ
= [X1|X2]
[β1
β2
]+ ǫ
= X1β1 + X2β2 + ǫ
◮ The Reduced model is
Y = X1β1 + ǫ
◮ Difference is term X2β2 which is 0 IF the null hypothesisHo : β2 = 0 is true.
Richard Lockhart STAT 350: F tests
Dimensions
◮ β has p parameters.
◮ βi has pi parameters with p1 + p2 = p.
Richard Lockhart STAT 350: F tests
Testing Ho : β2 = 0
◮ Fit both full and reduced models; get:
Y = µ̂F + ǫ̂F
= µ̂R + ǫ̂R
◮ Subscript F refers to full model and R to reduced model.
◮ Get decomposition of data vector Y into sum of threeperpendicular vectors:
Y = µ̂R + (µ̂F − µ̂R) + ǫ̂F
◮ I showed
µ̂R ⊥ µ̂F − µ̂R
µ̂R ⊥ ǫ̂F
µ̂F − µ̂R ⊥ ǫ̂F
Richard Lockhart STAT 350: F tests
Resulting ANOVA table
Source Sum of Squares Degrees of Freedom
X1 ||µ̂R ||2 p1
X2|X1 ||µ̂F − µ̂R ||2 p2
Error ||ǫ̂||2 n − p
Total (Unadjusted) ||Y ||2 n
Richard Lockhart STAT 350: F tests
F tests again
◮ In this table the notation X2|X1 means X2 adjusted for X1 orX2 after fitting X1.
◮ This table can now be used to test Ho : β2 = 0 by computing
F =MS(X2|X1)
MSE=||µ̂F − µ̂R ||2/p2
||ǫ̂||2/(n − p)
◮ Get P value from Fp2,n−p distribution.
◮ P value usually computed by software.
◮ Do level α test by comparing P to α.
Richard Lockhart STAT 350: F tests
Another Formula for this F statistic
◮ Recall that||µ̂R ||2 + ||ǫ̂R ||2 = ||Y ||2
and||µ̂R ||2 + ||µ̂F − µ̂R ||2 + ||ǫ̂F ||2 = ||Y ||2
so that||ǫ̂R ||2 = ||ǫ̂F ||2 + ||µ̂F − µ̂R ||2
◮ This makes
F =(ESSR − ESSF )/p2
ESSF/(n − p)
=ExtraSS/p2
ESSF/(n − p)
Richard Lockhart STAT 350: F tests
Remarks
1. If the errors are normal then
ESSF
σ2∼ χ2
n−p
2. If the errors are normal and Ho : β2 = 0 is true then
ExtraSSσ2
∼ χ2p2
3. ESSF is independent of the Extra SS.
4. SO:ExtraSS/(σ2p2)
ESSF /(σ2(n − p))= F ∼ Fp2,n−p
Richard Lockhart STAT 350: F tests
Example of the above: Multiple Regression
◮ Hardness, Yi , of plaster measured for each of 9 combinationsof sand content and fibre content.
◮ Sand content si was set at one of 3 levels as was fibre contentfi .
◮ All possible combinations tried, each on two batches ofplaster.
◮ Here is an excerpt of the data:
Sand Fibre Hardness Strength0 0 61 340 0 63 16
15 0 67 3615 0 69 1930 0 65 28
...
Richard Lockhart STAT 350: F tests
Models Fitted
◮ I fit submodels of the following ”Full” model:
Yi = β0 + β1si + β2s2i + β3fi + β4f
2i + β5si fI + ǫi
◮ There are 25 = 32 possible submodels of the full model
◮ Many of these 32 models are not sensible, such as
Yi = β0 + β4f2i + ǫi
orYi = β0 + β5si fi + ǫi
◮ Assume interaction term (β5si fi) is negligible unless each of Sand F have some effect .
◮ Assume that quadratic terms will probably not be presentunless linear terms are present.
◮ This limits the set of potential reasonable models.
◮ Fit each; error sum of squares in following table:
Richard Lockhart STAT 350: F tests
Fitting various models
Model for µ Error SS Error dfFull 81.264 12
β0 + β1si + β2s2i + β3fi + β4f
2i 82.389 13
β0 + β1si + β2s2i + β3fi 104.167 14
β0 + β1si + β2s2i 169.500 15
β0 + β1si 174.194 16β0 + β1si + β3fi + β4f
2i 87.083 14
β0 + β1fi + β2f2i 189.167 15
β0 + β1fi 210.944 16β0 + β1si + β3fi 108.861 15
Richard Lockhart STAT 350: F tests
Hypotheses tested
◮ First question: do 2nd degree polynomial terms, that is, thoseinvolving β2, β4 and β5 need to be included?
◮ Compare top line with model containing only β0 + β1si + β3fi .
◮ The extra SS is 108.861-81.264 on 3 degrees of freedomwhich gives a mean square of (108.861-81.264)/3= 9.199.
◮ The MSE is 81.264/12 = 6.772.
◮ Gives an F -statistic of 9.199/6.772=1.358 on 3 numeratorand 12 denominator degrees of freedom.
◮ P-value is 0.30 which is not significant.
◮ So we delete quadratic terms and consider the coefficients ofS and F .
Richard Lockhart STAT 350: F tests
A t test and an F test
Q : Are the effects of S and F additive?
A : Test Ho : β5 = 0.
◮ There are two methods to carry out such a test:
1. A t test2. A F test.
Fact : the F test is equivalent to a two sided t test.
◮ The t test uses
t =β̂5 − 0
σ̂β̂5
=β̂5√
MSE√
(XT X )−166
∼ t1,n−6
Richard Lockhart STAT 350: F tests
t tests
◮ See Distribution Theory section for linear combinations:
β5 = [0, 0, 0, 0, 0, 1]︸ ︷︷ ︸xT
β0
β1...
β5
and xT (XT X )−1x is the lower right hand corner entry in(XTX )−1, that is, (XTX )−1
66 .
◮ The F test uses
F =(ESSR − ESSF )/1
ESSFULL/(n − 6)∼ F1,n−6 (= t2)
Richard Lockhart STAT 350: F tests
Testing for Main Effects
The Data
◮ Y = hardness of plaster. n = 18 batches.
◮ S = sand content. Values used 0%, 15% 30%.
◮ F = fibre content. Values used 0%, 25% 50%.
◮ Factorial design with 2 replicates.
Richard Lockhart STAT 350: F tests
Comparison of Models“Full” model
Yi = β0 + β1si + β2s2i + β3fi + β4f
2i + β5si fI + ǫi
Fitted Models, ESS, df for error
Model for µ ESS Error dfFull 81.264 12β0 + β1si + β2s
2i + β3fi + β4f
2i 82.389 13
β0 + β1si + β2s2i + β3fi 104.167 14
β0 + β1si + β2s2i 169.500 15
β0 + β1si 174.194 16β0 + β1si + β3fi + β4f
2i 87.083 14
β0 + β1fi + β2f2i 189.167 15
β0 + β1fi 210.944 16β0 + β1si + β3fi 108.861 15β0(empty model) 276.278 17
Richard Lockhart STAT 350: F tests
Hypothesis tests:
1. Quadratic terms needed? Ho : β2 = β4 = β5 = 0.
◮ Extra SS = 108.861-81.264.◮ F = [(108.861− 81.264)/3]/[81.264/12] = 1.358.◮ Degrees of freedom are 3, 12 so P = 0.30, not significant.
Richard Lockhart STAT 350: F tests
3. Linear terms needed? There are several possible F -tests.
3.1 Compare full model to empty model.
F = (276.278− 81.264)/5/(81.264/12) = 5.76
so P is about .006.3.2 Assume full model is now additive, linear model
β0 + β1si + β3fi .
Then
F = [(276.278− 108.861)/2]/[108.861/15] = 11.53
and P is about 0.0009.3.3 Use estimate of σ2 from full model3.4 But get extra SS from last comparison:
F = [(276.278− 108.861)/2]/[81.264/12] = 12.36 for a Pvalue of 0.001
Richard Lockhart STAT 350: F tests
Top Related