one-way ANOVA

ANOVA

Requires:

1. All conditions contain independent samples
2. the dependent scores are normally distributed interval or ratio scores
3. the variances of the populations are homogenous

$n$ of each condition doesn’t need to be equal, but it’s way easier if they are. It only tests two-tailed hypothesis, but it’s actually a one-tailed test due to 0 being the same.

Diagram of an example study

	Factor A: Independent variable of perceived difficulty
Level $A_1$: Easy	Level $A_2$: Medium	Level $A_3$: Difficult
X	X	X
X	X	X
$\overline{X_1}$	$\overline{X_2}$	$\overline{X_3}$

k = 3 b/c there are 3 conditions.

This is similar to the two-sample t-test, but we can’t just do a bunch of pair-wise t-tests because the probability of making a Type I error is too high. This is because we’re doing multiple comparisons with a 0.05 margin for error.. which aggregates. ANOVA limits Type I probability to $\alpha$.

$$\begin{array}{rl}{s}_x^2& =\frac{\Sigma (X-\bar{X}{)}^2}{N-1}\\ & \\ & =\frac{\text{sum of squares}}{\text{degrees of freedom}}\\ & \\ & =\frac{\text{SS}}{\text{df}}\\ & \\ & =\text{mean square aka MS}\end{array}$$

“sum of squares” or “SS” is really short for “sum of the squared deviations”

Definitions

experiment-wise error rate : The probability of making a Type I error somewhere among the comparisons in an experiment.

Tukey’s HSD test : HSD = Honestly Significant Difference is a post-hoc procedure done after ANOVA to compare means between factors when all levels have equal $n$‘s.

Example

$H_0={\mu }_1={\mu }_2={\mu }_3=...={\mu }_k$. But $H_a:$ not all $\mu$ are equal.

So given the above table:

easy	medium	difficult
9	4	1
12	6	3
4	8	4
8	2	5
7	10	2	totals
sum(X): 40	30	15	85
sum(X^2): 354	220	55	629
n: 5	5	5	15
xbar: 8	6	3	k=3

Calculate the total sum of squares:

$$\begin{array}{rl}{SS}_{tot}& =\Sigma X_{tot}^2-\frac{(\Sigma X_{tot}{)}^2}{N}\\ & \\ & =629-\frac{85^2}{15}\\ & \\ & =629-\frac{7225}{15}\\ & \\ & =629-481.67\\ & \\ & =147.33\end{array}$$

Then calculate the sum of squares between groups

$$\begin{array}{rl}{SS}_{bn}& =\Sigma (\frac{(\Sigma X\text{ in column}{)}^2}{n\text{ in column}})-\frac{(\Sigma X_{tot}{)}^2}{N}\\ & \\ & =(\frac{40^2}{5}+\frac{30^2}{5}+\frac{15^2}{5})-\frac{(85{)}^2}{5}\\ & \\ & =(320+180+45)-481.67\\ & \\ & =545-481.67\\ & \\ & =63.33\end{array}$$

Then calculate the sum of squares within groups:

$$\begin{array}{rl}{SS}_{wn}& ={SS}_{tot}-{SS}_{bn}\\ & \\ & =147.33-63.33\\ & \\ & =84.00\end{array}$$

Compute the degrees of freedom:

$$\begin{array}{rl}{df}_{bn}& =k-1\\ & \\ & =3-1\\ & \\ & =2\\ & \\ & \\ & \\ {df}_{wn}& =N-k\\ & \\ & =15-3\\ & \\ & =12\\ & \\ & \\ & \\ {df}_{tot}& =N-1\\ & \\ & =15-1\\ & \\ & =14\end{array}$$

Then get the mean square between groups (${MS}_{bn}=\frac{{SS}_{bn}}{{df}_{bn}}$):

${MS}_{bn}=\frac{63.33}{2}=31.67$

Within groups (${MS}_{wn}=\frac{{SS}_{wn}}{{df}_{wn}}$):

${MS}_{wn}=\frac{84}{12}=7$

then: $F_{obt}=\frac{{MS}_{bn}}{{MS}_{wn}}=\frac{31.67}{7}=4.52$

Which leaves us with:

Source	Sum of squares	df	mean square	f_obt
between	63.33	2	31.67	4.52
within	84	12	7
total	147.33	14

Distribution

Unlike t and z distribution, the f-distribution is positively skewed, b/c there’s no upper limit for how big f can be.. but it can never be lower than 0. Unlike those other distributions, finding $F_{crit}$ requires the df for both between and within to look up in the “F-table”.

For the example above, we get $F_{crit}=3.88$, so this is sufficient evidence to reject the null hypothesis.

This leads us to conclude that there does appear to be a relationship between perceived difficulty and score. We don’t know if it applies to all columns, though.

Post-hoc test

Tukey’s HSD is $HSD=(q_k)(\sqrt{\frac{{MS}_{wn}}{n}})$ $q_k$ is found in a table called “Values of Studentized Range Statistic” (table 5 in the appendix of Behavioral Sciences Stats). For $k=3,{df}_{wn}=12,\alpha =.05$, $q_k=3.77$.

$$\begin{array}{rl}HSD& =(q_k)(\sqrt{\frac{{MS}_{wn}}{n}})\\ & \\ & =(3.77)(\sqrt{\frac{7}{5}})\\ & \\ & =3.77(\sqrt{1.4})\\ & \\ & =3.77(1.183)\\ & \\ & =4.46\end{array}$$

Get all the differences of level-mean combinations:

x1 = 8 ; aka easy x2 = 6 ; aka medium x3 = 3 ; aka hard

x1 - x2 = 2 x1 - x3 = 5 x2 - x3 = 3

Compare to HSD. If the absolute difference is greater than the HSD, then they have signifiant differences. (so easy -> hard was a significant difference)