Hello, dear friend, you can consult us at any time if you have any questions, add WeChat: zz-x2580
One-way and two-way ANOVA models
STAT3022
One-way ANOVA model
Setup
• Comparing the mean response (Y ) of t different treatments,
when it is reasonable to assume that the treatment only
affects the response via the mean.
• Data: yij , i = 1, . . . , t, j = 1, . . . , ni, i.e there are ni
observations receiving treatment i. Total number of
observations is n =
∑t
i=1 ni.
• yi• =
ni∑
j=1
yij , y¯i• = n−1i yi• : sum and average of response
for treatment i.
• y•• =
t∑
i=1
yi• =
t∑
i=1
ni∑
j=1
yij , y¯•• = n−1y•• : sum and average
of all the responses.
1
Factor-effect model
To study the effect of treatments, we use the factor-effect model
yij = µ+ αi + εij , εij ∼ N(0, σ2), i = 1, . . . , t, j = 1, . . . , ni
• µ represents the overall mean.
• αi represents the difference between mean of treatment i and
overall mean. Together µi = µ+ αi represents the mean of
treatment group i.
• εij is the model error term.
2
Factor-effect model
The factor-effect model is a linear model with
β(t+1)×1 = (µ, α1, · · · , αt)>,
yn×1 = (y11 · · · , y1n1 , y21, · · · , ytnt)>,
Xn×(t+1) =
1 1 0 · · · 0
1 1 0 · · · 0
...
...
...
. . .
...
1 1 0 · · · 0
1 0 1 · · · 0
...
...
...
. . .
...
1 0 0 · · · 1
.
Note that rank(X) = t < (t+ 1) since the first column is equal to
the sum of all remaining columns.
3
Least square estimators
Normal equations: X>X βˆ = X> y, i.e
n n1 · · · nt
n1 n1 0 · · · 0
...
...
. . .
...
nt 0 · · · 0 nt
µˆ
αˆ1
...
αˆt
=
y••
y1•
...
yt•
,
nµˆ+
t∑
i=1
niαˆi = y••, niµˆ+ niαˆi = yi•, i = 1, . . . , n
Without any constraint, the system has an infinite number of
solution. The two most common constraints are:
• Sum constraint:
∑t
i=1 niαˆi = 0, or
• First treatment constraint: αˆ1 = 0.
4
Different constraints lead to different estimates
• Under sum constraint, we have nµˆ = y••, so µˆ = y¯••, then
αˆi = y¯i• − y¯••, i = 1, . . . , n.
• Under first treatment constraint, we have µˆ = y¯1•, and
αˆi = y¯i• − y¯1• for i ≥ 2.
• The first treatment constraint is default in R.
5
An example
Does caffeine stimulation affect the rate at which individuals can
tap their fingers?
• n = 30 male students randomly allocated to t = 3 groups of
10 students each:
Group 1: zero caffeine dose
Group 2: low caffeine dose
Group 3: high caffeine dose
• Two hours after treatment, each subject tapped fingers as
quickly as possible for a minute. Number of taps is recorded.
6
An example
242.5
245.0
247.5
250.0
252.5
High Low Zero
Dose
Ta
ps
7
Estimable linear combination
• Although different constraints lead to different estimates of β,
there exist a class of linear combinations of β that are
uniquely determined, which we call them to be estimable.
• We will use the following result without proving it.
Let θ = a>β denote a linear combination of coefficients in
a linear model y = Xβ + ε, where the design matrix X has
dimension n × p and where a is a p × 1 row vector. Hence,
θ is estimable if and only if there exist another vector c of
dimension n × 1 such as a> = c>X; in other words, a has
to be a linear combination of row vectors of X.
Fortunately, most useful linear combinations are estimable.
8
Estimable linear combination
X =
1 1 0 · · · 0
...
... · · · ...
1 1 0 · · · 0
1 0 1 · · · 0
...
... · · · ...
1 0 0 · · · 1
rm. dup−→ X˜t×(t+1) =
1 1 0 . . . 0
1 0 1 . . . 0
...
... · · · ...
1 0 0 . . . 1
For θ = a>β to be estimable, then a> = c>X. Denoting elements
of c as ci with i = 1, . . . , n, that implies a
> has the form:
a> =
[∑t
i=1 ci c1 · · · ct
]
9
Estimable linear combination
Noting that β = (µ, α1, . . . , αt)
>, so the estimable linear
combination has the form
θ = a>β =
(
t∑
i=1
ci
)
µ+
t∑
i=1
ciαi.
Below are the most important estimable linear combinations:
• Group mean µi = µ+ αi (ci = 1 and cj = 0 for all i 6= j).
• Differences between groups αi − αj (ci = 1, cj = −1, ck = 0
for all k 6= i, j)
• In general, any contrast
∑t
i=1 ciαi, with
∑t
i=1 ci = 0.
10
Estimable linear combination
• When a linear combination is estimable, then we can use any
a>βˆ to estimate it, with βˆ is the solution of the normal
equation (regardless of the constraints)
• Example: for the group mean
µˆi = µˆ+ αˆi = y¯•• + (y¯i• − y¯••) (under sum constraint)
= y¯1• + (y¯i• − y¯1•) (under 1st treatment constraint)
= y¯i•.
• Verify that in both sum and first treatment constraint, the
mean difference between group i and group j is estimated by
αˆi − αˆj = y¯i• − y¯j•.
11
Partitioning of variability
Similar to the multiple linear regression model case, we also have
the following partitioning of variability
