Hypothesis Testing

statistics

statistical hypothesis : a statement about the numerical value of a population parameter

null hypothesis : the status quo; assumed to be true unless there’s convincing data otherwise. Denoted $H_0$

alternative hypothesis : hypothesis that’s only accepted if there’s compelling evidence, denoted $H_a$

test statistic : sample statistic, computed from information in the sample, used to decide between null and alternative hypothesis.

Type I error : (relationship doesn’t actually exist); researcher rejects the null hypothesis in favor of the alternative, when $H_0$ is true. The probability of committing Type I error is denoted $\alpha$. aka false positive

Type II error : (relationship really does exist); when $H_0$ is accepted, but it’s actually false. The probability of this error is denoted by $\beta$. aka false negative

Power (statistics)- rejection region :: the possible values where the researcher will reject $H_0$ in favor of $H_{\alpha }$.

criterion : probability that defines whether it’s unlikely to represent the underlying population.

critical value : the score that marks the inner edge of the region of rejection

one-tailed test : the alternative case is strictly greater than the specific value (e.g. “strength of pipe must be > 2400 psi”)

two-tailed test : alternative hypothesis exists within some bounds (e.g. tolerance of a machined part)

p-value : aka observed significance level, is the probability of observing a value of the test statistic that is at least as contradictory to the null hypothesis and supportive of the alternative hypothesis, as the actual one computed form the sample data. Reworded: The likelihood that a test statistic overlaps with an interval, given a confidence value.

power of the test : probability that the test will correctly lead to the rejection of the null hypothesis for a particular value of $\mu$ or $p$ in the alternative hypothesis. The power is equal to $(1-\beta )$.

significant : It means that this result is unlikely to have occured due to a sampling error; Indicates a rejection of the null hypothesis $H_0$

significance level : This is $\alpha$. $1-\alpha$ is the confidence level.

inferential statistics : procedures for deciding whether sample data represent a particular relationship in the population

parametric statistics : inferential stats for computing the mean; Require certain assumptions about the raw score population represented by the sample

nonparametric statistics : used for median and mode; don’t require assumptions about the raw score population of the sample

When a sample statistic exists outside of the “rejection region”, we can’t reject the null hypothesis.. but rather say “the sample evidence is insufficient to reject $H_0$ at $\alpha =.05$“.

Elements of a hypothesis test

1. Null hypothesis ($H_0$).
2. Alternative (research) hypothesis ($H_{\alpha }$).
3. Test statistic.
4. Rejection region, which uses $\alpha$ (also referred to as the “level of significance”).
5. Assumtpions: clear statements about the population being sampled.
6. Experiment & calculation of test statistic: the computation of the test statistic
7. Conclusion: rejection (with possible type I error) if the value is in the rejection region. Insufficient evidence to reject if it isn’t (given we don’t know the probability of $\beta$, which is the likelihood of type II error).

Formulating a hypothesis

1. Pick an alternative hypothesis
   1. upper-tailed ($H_a:\mu >2400$)
   2. lower-tailed ($H_a:\mu <2400$)
   3. two-tailed ($H_a:\mu \ne 2400$)
2. select null hypothesis ($H_0:\mu =2400$).

Calculating a p-value

1. Determine the value of the test stat $z$ corresponding with the result of the sampling experiment.
2. If the test is one-tailed, the p-value is the area above or below (depending on which tail) the observed z-value. If it’s two-tailed, p-value is 2x the tail area beyond the z-value in the direction of z.

In practice, this means that you take the CDF of the z value (or perhaps, 1-z value, depending on direction) and check if it’s smaller than the $\alpha$ you’ve chosen. If so, you should reject the null hypothesis, b/c it lives within the rejection region.

Calculating a p-value from a z score

You just need to take the cdf of the z-score. Ensure you’re capturing the tail(s) that you care about.

Converting a two-tailed p-value to a one-tailed p-value

$$\begin{array}{r}p=\frac{\text{reported p-value}}{2}\text{if}\{\begin{array}{l}{H}_a\text{ is of the form}>\text{ and }z\text{ is positive }\\ \\ H_a\text{ is of the form}<\text{ and }z\text{ is negative }\end{array}\\ \end{array}$$ $$\begin{array}{r}p=1-(\frac{\text{reported p-value}}{2})\text{if}\{\begin{array}{l}{H}_a\text{ is of the form}>\text{ and }z\text{ is negative }\\ \\ H_a\text{ is of the form}<\text{ and }z\text{ is positive }\end{array}\end{array}$$

p-value for proportions / chi

1. draw curve
2. draw region
3. label areas / test stat
4. compute the p-value

to compute the p-value, compute the chi-square CDF for the null hypothesis.

Statistical tests

Statistical tests often take the form of:

(observed difference - what we expect if null is true) / average variation

Z statistic testing

Statistical tests Used when you have a large (n>=30) sample Test statistic: $z_c=\frac{\bar{x}-{\mu }_0}{\sigma /\sqrt{n}}$ Rejection region depend on alternative hypothesis.

• $H_a:\mu <{\mu }_0,z_c<-z_{\alpha }$
• $H_a:\mu >{\mu }_0,z_c>z_{\alpha }$

This requires:

1. a random sample
2. the sample size is large enough ($n\ge 30$)

T statistic testing

Statistical tests Used for small samples Test statistic: $t_c=\frac{\bar{x}-{\mu }_0}{s/\sqrt{n}}$

Rejection region depend on alternative hypothesis.

• $H_a:\mu <{\mu }_0,t_c<-t_{\alpha }$
• $H_a:\mu >{\mu }_0,t_c>t_{\alpha }$

This requires:

1. A random sample
2. The population is approximately normal

Hypothesis about a population proportion

Test Statistic: $z=\frac{\hat{p}-p_0}{\sqrt{p_0{q}_0/n}}$ Rejection region depend on alternative hypothesis.

• $H_a:p<p_0,z_c<-z_{\alpha }$
• $H_a:p>p_0,z_c>z_{\alpha }$

This requires:

1. A random sample of a binomial population
2. Sample size is n large ($n{p}_0\ge 15,n{q}_0\ge 15$)

Calculating $\beta$ for a mean

1. Calculate $\bar{x}$ that corresponds to the border between acceptance/rejection regions, denoted ${\bar{x}}_0$. ${\bar{x}}_0={\mu }_0+z_{\alpha }{\sigma }_{\bar{x}}$ for upper tailed tests. (minus for lower, and both plus and minus w/ $\frac{\alpha }{2}$ instead for two tailed)
2. Specify the value of ${\mu }_a$ in the alternative hypothesis to calculate for $\beta$. Convert ${\bar{x}}_0$ to z-values, using the alternative. $z=\frac{{\bar{x}}_0-{\mu }_a}{{\sigma }_{\bar{x}}}$.

Example

$H_0:\mu =2400,H_{\alpha }:\mu >2400$ test statistic: $z=\frac{\bar{x}-2400}{\sigma /\sqrt{n}}$ Rejection region: $z>1.645$ for $\alpha =.05$. (note: this is only one tailed) s: 200 n: 50

—

$$\begin{array}{r}\bar{x}={\mu }_0+1.645{\sigma }_{\bar{x}}\\ \\ =2400+1.645(\frac{\sigma }{\sqrt{n}})\\ \\ \approx 2400+1.645(\frac{s}{\sqrt{n}}\\ \\ =2400+1.645(\frac{200}{\sqrt{5}0})\\ \\ =2400+1.645(28.28)\\ \\ =2446.5\end{array}$$

Then find the z-value for ${\bar{x}}_0$ (the border between rejection/acceptance).

Unsure: Where does 2425 come from? Do you just pick it out of a hat b/c it’s in the alternative hypothesis?

$$\begin{array}{r}z=\frac{{\bar{x}}_0-2425}{{\sigma }_{\bar{x}}}\\ \\ \approx \frac{{\bar{x}}_0-2425}{s/\sqrt{n}}\\ \\ =\frac{2446.5-2425}{28.28}\\ \\ =.76\end{array}$$

so z = .76. The area under the curve is $\beta =.5+.2764=.7764$.

This B value comes from Table II of our book, which gets the area under the curve. Likely need to +.5 if it’s bigger than the mean.

Calculating $\beta$ for a p (proportion)

Same as above, but we use $\sqrt{\frac{p_0{q}_0}{n}}$ instead of ${\sigma }_{\bar{x}}$ aka $\frac{s}{\sqrt{n}}$

Hypothesis about population variance

NOTE: Be careful on if you’re talking about variance or stddev and adjust accordingly.

test statistic: ${\chi }_a^2=\frac{(n-1)s^2}{{\sigma }_0^2}$

Assumes:

1. random sample
2. population is approximately normal

For CVA (common value approach), we get the critical values by doing an inverse chi-square passing alpha and 1-alpha (assuming 2 tailed).

For PVA (p value approach): It’s the chi-cdf of the test stat, accounting for tail’dness.