100  When to use a one-sided or two-sided test?

The choice between one-sided and two-sided tests must be made before inspecting the sample results. In addition, most statistical software defaults to two-sided tests, so the analyst must explicitly justify any one-sided choice.

Generally speaking, the key factor is whether there is strong, theory-based prior justification about the direction of the population effect.

If, based on prior knowledge, only positive deviations are scientifically meaningful, use a one-sided (right-tailed) test.

If, based on prior knowledge, only negative deviations are scientifically meaningful, use a one-sided (left-tailed) test.

In all other cases, use a two-sided test. A two-sided test is also the safe default when directional prior knowledge is weak or absent.

Changing from two-sided to one-sided after seeing the sample mean inflates the type I error and should be avoided.

100.1 Worked Example

Suppose you test \(H_0: \mu = 50\) with observed test statistic \(z_{obs} = 2.05\).

• Two-sided p-value: \(p_{2s} = 2(1-\Phi(2.05)) \approx 0.040\).
• Right-sided p-value: \(p_{1s} = 1-\Phi(2.05) \approx 0.020\).

At \(\alpha = 0.05\), both tests reject \(H_0\) in this case.

Now consider a weaker signal, e.g. \(z_{obs}=1.80\):

• Two-sided: \(p_{2s} \approx 0.072\) (fail to reject).
• Right-sided: \(p_{1s} \approx 0.036\) (reject).

This illustrates why the direction must be pre-specified. If the right-sided test was chosen only after seeing that \(z_{obs}>0\), the inference is biased and the actual Type I error is inflated (for symmetric tests, a nominal one-sided level \(\alpha\) becomes about \(2\alpha\)).