116  Paired Two Sample t-Test

The Paired Two Sample t-Test is sometimes called “Dependent Samples t-Test” or “Repeated Measures t-Test”. When pairing is valid and within-pair correlation is positive, this test typically has greater power (smaller type II error \(\beta\)) than the Unpaired Two Sample t-Test. In essence, this test is treated as a One Sample t-Test where each of the \(n\) paired differences is treated as one observation.

116.1 Hypotheses -- Examples

Suppose we wish to test the following two-sided statistical hypothesis for a bivariate, quantitative dataset:

\[ \begin{cases}\text{H}_0: \mu_1 - \mu_2 = \mu_0 \\\text{H}_A: \mu_1 - \mu_2 \neq \mu_0\end{cases} \]

where \(\mu_0 = 0\) which is equivalent to testing

\[ \begin{cases}\text{H}_0: \mu_1 = \mu_2 \\\text{H}_A: \mu_1 \neq \mu_2\end{cases} \]

In case we wish to perform a one-sided test, we can formulate the following hypotheses:

\[ \begin{cases}\text{H}_0: \mu_1 - \mu_2 \leq \mu_0 \\\text{H}_A: \mu_1 - \mu_2 > \mu_0 \end{cases} \]

or

\[ \begin{cases}\text{H}_0: \mu_1 - \mu_2 \geq \mu_0 \\\text{H}_A: \mu_1 - \mu_2 < \mu_0 \end{cases} \]

The underlying theory is described in Chapter 109 (Statistical Test of the difference between Means -- Dependent/Paired Samples). The chosen type I error \(\alpha\) is 5%.

116.2 Analysis based on p-values and confidence intervals

The software can be found on the public website (https://compute.wessa.net/rwasp_twosampletests_mean.wasp) and on RFC (“Hypotheses / Empirical Tests”).

116.2.1 Data & Parameters

This R module contains the following fields:

  • Data X: a multivariate dataset containing quantitative data
  • Names of X columns: a space delimited list of names (one name for each column)
  • Column number of first sample: a positive integer value of the column in the multivariate dataset which corresponds to the first sample
  • Column number of second sample: a positive integer value of the column in the multivariate dataset which corresponds to the second sample
  • Confidence: this is \(1 - \alpha\) (i.e. 1 minus the chosen type I error)
  • Alternative: parameter which defines the type of Hypothesis Test to be computed. This parameter can be set to the following values:
    • two.sided
    • less
    • greater
  • Are observations paired?: This parameter can be set to the following values:
    • unpaired
    • paired
  • Null Hypothesis: this is the value of \(\mu_0\) against which the hypothesis is tested (often it is this case that \(\mu_0 = 0\))

116.2.2 Output

The following analysis focuses on two psychometric, academic motivation constructs IM.Know and IM.Accomplishment from a large student sample:

Interactive Shiny app (click to load).
Open in new tab

The p-value for the two-sided hypothesis is smaller than the chosen type I error. As a consequence we reject the Null Hypothesis.

The same conclusion can be drawn from the two-sided confidence interval, i.e. \(\mu_0 = 0 \notin [1.657253,2.093908]\).

When we specify Alternative = "less", we are testing \(\text{H}_A: \mu_1 - \mu_2 < \mu_0\).

Choosing Alternative = "greater" yields a p-value < 2.2e-16 which is smaller than the chosen type I error \(\alpha = 0.05\). As a consequence we reject the Null Hypothesis.

The one-sided direction (less or greater) must be chosen a priori based on theory or design, not selected after observing sample means.

The left-sided confidence interval allows us to draw the same conclusion because \(\mu_0 = 0 \notin [1.692403, \infty]\), which implies that we should reject the Null Hypothesis.

For reporting, include a paired-effect size (Cohen 2013) such as

\[ d_z = \frac{\bar{d}}{s_d}, \]

where \(\\bar{d}\) and \(s_d\) are the mean and standard deviation of paired differences.

To compute the Paired Two Sample t-Test on your local machine, the following script can be used in the R console.

Note: this local script is a synthetic template. The embedded app example above uses the AMS dataset and therefore has different numeric output.

set.seed(123)
A <- rnorm(150)
B <- rnorm(150)
x <- cbind(A, B)
par1 = 1 #column number of first sample
par2 = 2 #column number of second sample
par3 = 0.95 #confidence (= 1 - alpha)
par4 = 'two.sided'
par5 = 'paired'
par6 = 0.0 #Null Hypothesis
if (par5 == 'unpaired') paired <- FALSE else paired <- TRUE
(t.test(x[,par1], x[,par2], alternative=par4, paired=paired, mu=par6, conf.level=par3))

    Paired t-test

data:  x[, par1] and x[, par2]
t = -0.99818, df = 149, p-value = 0.3198
alternative hypothesis: true mean difference is not equal to 0
95 percent confidence interval:
 -0.3504276  0.1152105
sample estimates:
mean difference 
     -0.1176086

116.3 Assumptions

Since we treat each pair of observations as one effective measurement, the assumptions of the Paired Two Sample t-Test are the same as for the One Sample t-Test (Section 114.4).

116.4 Alternatives

Again, the alternatives are the same as for the One Sample t-Test (Section 114.5).