t-tests

They’re like z-tests, but for when we don’t know the population standard deviation.

Definitions

independent-samples t-test : parametric procedure used to test sample means from two independent samples

independent samples : samples created by selecting each participant for one condition without regard to the participants selected for any other condition

homogeneity of variance : the requirement that the populations represented in a study have equal variances (${\sigma }_x^2$)

sampling distribution of differences between means : infinitely take samples from one population then plot the frequency distribution of their differed means (${\bar{X}}_1-{\bar{X}}_2$). In this distribution, $\mu$ is 0 (e.g. they’re equal)

pooled variance : Weight average of the sample variances in a two-sample t-test

effect size : the amount of influence that changing the conditions of the independent variable had on the dependent scores

Cohen’s d : A measure of effect in a two-sample experiment that shows the magnitude of the differences between the means

square point-biserial correlation coefficient : $r_{pb}^2$ proportion of variance in dependent scores that is accounted for by the independent variable in a two-sample experiment

One sample t-test

A one-sample t-test assumes you have a one-sample experiment using interval or ratio scores.

Because the t-distribution changes shape based on the size of samples used (n), this changes what x value $\alpha$ is at (e.g. it’s not just 1.96/1.645). You can look this value up in a t-table using the degrees of freedom (df). When df>=120, it’s basically the Normal Distribution.

When df gets large, they start bucketing the critical values into ranges (e.g. df=65, but you’ll only see df=60 & df=120). If it’s less than the small number (2.00 at $\alpha =.05$), you’re not significant. If it’s bigger than the big number (1.98), you are significant. If it’s somewhere in the middle, either use software like JASP/SPSS or do something called linear interpolation.

Steps:

1. Create a 1 or 2 tailed $H_0$ and $H_a$.
2. Compute $t_{obt}$
   1. Compute $\bar{x}$ and $s_x^2$
   2. Compute $s_{\bar{x}}$
   3. Compute $t_{obt}$
3. Use $df=N-1$ to find $t_{crit}$ in the t-table

Another way to think of $t_{obt}$:

from math import sqrt
 
def t_obt(xbar, s2x,n,u):
  sxbar = sqrt(s2x/n)
  return (xbar-u)/sxbar

Determining population parameter $\mu$ w/ interval estimation

We can do this two ways: “point estimation” allows us to say mu is equal to our sample mean, but that is subject to sampling error.

Instead, we can use interval estimation, which results in a statement plus a margin of error. We calculate this by determining the highest likely value (i.e. $t_{crit}$) that would still be represented by our sample mean. Because we want the upper and lower bounds, we always use the two-tailed version of $t_{crit}$.

The formula for confidence interval

$$\begin{array}{r}{s}_{\bar{x}}(-t_{crit})+\bar{x}\le \mu \le s_{\bar{x}}(+t_{crit})+\bar{x}\end{array}$$

Example

We are testing the optimisim scores for a group of men (Behavioral Sciences Stats pg 129) and have the following data. We are trying to test if $H_0=(\mu =10)$

participants	scores (X)	X^2
1	9	81
2	8	64
3	10	100
4	7	49
5	8	64
6	8	64
7	6	36
8	4	16
9	10	100
sums	70	574
mean	7.7777778

We can then calculate the estimated Standard Error, but first we need the estimated population variance:

$$\begin{array}{rl}{s}_x^2& =\frac{\Sigma X^2-\frac{(\Sigma X{)}^2}{N}}{N-1}\\ & \\ & =\frac{574-\frac{(70{)}^2}{9}}{8}\\ & \\ & =\frac{574-\frac{4900}{9}}{8}\\ & \\ & =\frac{574-544.\bar{4}}{8}\\ & \\ & =\frac{29.5555}{8}\\ & \\ & =3.6944\end{array}$$

then:

$$\begin{array}{rl}{s}_{\bar{x}}& =\sqrt{\frac{s_x^2}{N}}\\ & \\ & =\sqrt{\frac{3.6944}{9}}\\ & \\ & =\sqrt{.4105}\\ & \\ & =.6407\end{array}$$

which we can then determine (note: mu is the thing we’re testing for in $H_0$; $t_{obt}$ is the value we found that we need to compare to the critical value to determine if it’s significant):

$$\begin{array}{rl}{t}_{obt}& =\frac{\bar{X}-\mu }{s_{\bar{x}}}\\ & \\ & =\frac{7.778-10}{.6407}\\ & \\ & =\frac{-2.222}{.6407}\\ & \\ & =-3.468\end{array}$$

given $n=9$ and $\alpha =.05$, $t_{crit}=2.306$ so there is sufficient evidence to reject the null hypothesis here.

Two sample t-test

We usually don’t know $\mu$, so research often uses the two sample t-test. We presume that in condition 1, ${\bar{x}}_1$ represents $\mu$ if you tested the population. Then, we can conduct an altered condition 2 to see how it changes relative to that presumptive $\mu$.

The two sample test comes in two varieties: independent-samples t-test and the related-samples t-test. Independent-samples might be: You got a bunch of rats, gave half food A and half food B. See if there was a difference.

Related samples: You got a bunch of rats. You fed them food A. Tested them. Then fed them food B. Tested again. The two samples (A & B) are related (b/c they’re the same rats).

Effect size

This is “cool. It changed something.. but like.. how much did it move the needle?”

We can compute this two ways:

• Cohen’s d (how much does this move the needle?)
• proportion of variance accounted for (does it have a consistent effect?)

Cohen’s D

This measures the size of the changes in the scores.

Independent tests use $d=\frac{\overline{X_1}-\overline{X_2}}{\sqrt{s_{pool}^2}}$ Related tests use $d=\frac{\bar{D}}{\sqrt{s_D^2}}$

Cohen proposed these guidelines:

d = .2	small effect
d = .5	medium effect
d = .8	large effect

Proportion of variance accounted for

This measures how consistently the scores change.

Given this data from a related-sample study:

Before therapy	After
10	5
11	6
12	7

We can see that there is some variance b/c of the independent variable (people b/c 10 != 11).

We describe this via “square point-biserial correlation coefficient”

if $t_{obt}=+2.93;df=30$,

$$\begin{array}{rl}{r}_{pb}^2& =\frac{t_{obt}^2}{t_{obt}^2+df}\\ & \\ & =\frac{2.93^2}{2.93^2+30}\\ & \\ & =\frac{8.585}{38.585}\\ & \\ & =.22\end{array}$$

Guidelines for interpreting:

< 0.9	Small effect
> .10 < .25	moderate / relatively common
> .25	large / rare

ANOVA

When calculating effect size for ANOVA, we’re looking for eta squared ${\eta }^2=\frac{S{S}_{bn}}{S{S}_{tot}}$.