120  Two One-Sided Tests (TOST) for Equivalence

120.1 Definition

The Two One-Sided Tests (TOST) procedure (Schuirmann 1987) is a method for testing whether two means are equivalent within a pre-specified margin. Unlike traditional hypothesis testing which tests for a difference, equivalence testing tests whether any difference is small enough to be considered negligible.

In traditional hypothesis testing (Chapter 104), the null hypothesis states that there is no difference between groups, and we reject this null when evidence suggests a difference exists. In equivalence testing, the logic is reversed: the null hypothesis states that the groups are not equivalent, and we reject this null when evidence suggests the groups are equivalent.

120.2 The Equivalence Margin

The equivalence margin \(\Delta\) (delta) defines the largest difference that is considered practically unimportant. If the true difference between means lies within the interval \([-\Delta, +\Delta]\), the two groups are considered equivalent.

The choice of \(\Delta\) must be made before data collection and should be based on:

  • Domain knowledge about what constitutes a meaningful difference
  • Regulatory requirements (e.g., bioequivalence studies typically use \(\Delta = 0.2\) or 20%)
  • Clinical or practical significance thresholds

120.3 Hypotheses

The TOST procedure decomposes the equivalence hypothesis into two one-sided hypotheses:

Test 1 (Lower bound):

\[ H_{01}: \mu_1 - \mu_2 \leq -\Delta \quad \text{vs.} \quad H_{a1}: \mu_1 - \mu_2 > -\Delta \]

Test 2 (Upper bound):

\[ H_{02}: \mu_1 - \mu_2 \geq +\Delta \quad \text{vs.} \quad H_{a2}: \mu_1 - \mu_2 < +\Delta \]

Equivalence is concluded if both null hypotheses are rejected at significance level \(\alpha\). This is equivalent to checking whether the \((1 - 2\alpha)\) confidence interval for the difference falls entirely within \([-\Delta, +\Delta]\).

120.4 Test Statistic

For each one-sided test, the test statistic follows the same form as the two-sample t-test (Chapter 118):

\[ t_1 = \frac{(\bar{x}_1 - \bar{x}_2) - (-\Delta)}{SE} = \frac{(\bar{x}_1 - \bar{x}_2) + \Delta}{SE} \]

\[ t_2 = \frac{(\bar{x}_1 - \bar{x}_2) - (+\Delta)}{SE} = \frac{(\bar{x}_1 - \bar{x}_2) - \Delta}{SE} \]

where \(SE\) is the standard error of the difference between means. For independent samples with equal variances:

\[ SE = s_p \sqrt{\frac{1}{n_1} + \frac{1}{n_2}} \]

where \(s_p\) is the pooled standard deviation.

120.5 Decision Rule

Equivalence is established if:

  • \(t_1 > t_{\alpha, df}\) (reject \(H_{01}\)), AND
  • \(t_2 < -t_{\alpha, df}\) (reject \(H_{02}\))

Alternatively, using p-values: equivalence is established if both \(p_1 < \alpha\) and \(p_2 < \alpha\).

The overall p-value for the TOST procedure is the larger of the two one-sided p-values:

\[ p_{TOST} = \max(p_1, p_2) \]

120.6 Confidence Interval Approach

An equivalent approach uses the \((1 - 2\alpha) \times 100\%\) confidence interval for the difference \(\mu_1 - \mu_2\):

\[ (\bar{x}_1 - \bar{x}_2) \pm t_{\alpha, df} \times SE \]

If this confidence interval lies entirely within \([-\Delta, +\Delta]\), equivalence is concluded at significance level \(\alpha\).

120.7 Example

A pharmaceutical company wants to demonstrate that a generic drug is bioequivalent to a brand-name drug. The equivalence margin is set at \(\Delta = 5\) units. Two groups of patients receive either the generic (\(n_1 = 24\)) or brand-name (\(n_2 = 24\)) drug, and a pharmacokinetic measure is recorded.

Results:

  • Generic: \(\bar{x}_1 = 102.3\), \(s_1 = 4.3\)
  • Brand: \(\bar{x}_2 = 100.1\), \(s_2 = 3.8\)
  • Difference: \(\bar{x}_1 - \bar{x}_2 = 2.2\)

The 90% confidence interval for the difference (using \(\alpha = 0.05\) for each one-sided test) is \([0.3, 4.2]\).

Since the entire 90% confidence interval lies within \([-5, 5]\), equivalence can be established at \(\alpha = 0.05\). The generic drug is shown to be bioequivalent to the brand-name drug based on this study.

120.8 R Module

120.8.1 Public website

The TOST procedure is available on the public website:

120.8.2 RFC

The TOST module is available in RFC under the menu “Hypothesis Testing / Equivalence Testing (TOST)”.

120.8.3 R Code

The following code demonstrates the TOST procedure using the TOSTER package:

install.packages("TOSTER")
library(TOSTER)

# Example data
generic <- c(98, 105, 102, 99, 108, 101, 95, 110, 103, 97,
             106, 100, 104, 99, 107, 102, 96, 109, 101, 98,
             105, 103, 100, 108)
brand <- c(96, 102, 100, 98, 105, 99, 94, 107, 101, 95,
           103, 98, 102, 97, 104, 100, 94, 106, 99, 96,
           103, 101, 98, 105)

# TOST procedure with equivalence margin of 5
result <- tsum_TOST(
  m1 = mean(generic),
  sd1 = sd(generic),
  n1 = length(generic),
  m2 = mean(brand),
  sd2 = sd(brand),
  n2 = length(brand),
  eqb = 5,           # equivalence bound (symmetric)
  alpha = 0.05
)

print(result)

Welch Two Sample t-test

The equivalence test was significant, t(45.39) = -2.398, p = 1.03e-02
The null hypothesis test was non-significant, t(45.39) = 1.897, p = 6.42e-02
NHST: don't reject null significance hypothesis that the effect is equal to zero 
TOST: reject null equivalence hypothesis

TOST Results 
                t    df p.value
t-test      1.897 45.39   0.064
TOST Lower  6.192 45.39 < 0.001
TOST Upper -2.398 45.39    0.01

Effect Sizes 
               Estimate     SE             C.I. Conf. Level
Raw              2.2083 1.1640 [0.2538, 4.1629]         0.9
Hedges's g(av)   0.5385 0.3003 [0.0596, 1.0117]         0.9
Note: SMD confidence intervals are an approximation. See vignette("SMD_calcs").

For a manual implementation without external packages:

# Manual TOST implementation
tost_test <- function(x1, x2, delta, alpha = 0.05) {
  n1 <- length(x1)
  n2 <- length(x2)
  m1 <- mean(x1)
  m2 <- mean(x2)

  # Pooled standard deviation
  sp <- sqrt(((n1 - 1) * var(x1) + (n2 - 1) * var(x2)) / (n1 + n2 - 2))
  se <- sp * sqrt(1/n1 + 1/n2)
  df <- n1 + n2 - 2

  # Difference
  diff <- m1 - m2

  # Two one-sided tests
  t1 <- (diff + delta) / se  # Test against lower bound
  t2 <- (diff - delta) / se  # Test against upper bound

  p1 <- pt(t1, df, lower.tail = FALSE)  # Upper tail for t1
  p2 <- pt(t2, df, lower.tail = TRUE)   # Lower tail for t2

  # Confidence interval (90% for alpha = 0.05)
  t_crit <- qt(1 - alpha, df)
  ci_lower <- diff - t_crit * se
  ci_upper <- diff + t_crit * se

  # Decision
  equivalent <- (p1 < alpha) & (p2 < alpha)

  list(
    difference = diff,
    se = se,
    t1 = t1, p1 = p1,
    t2 = t2, p2 = p2,
    p_tost = max(p1, p2),
    ci = c(ci_lower, ci_upper),
    delta = delta,
    equivalent = equivalent
  )
}

# Apply to example data
result <- tost_test(generic, brand, delta = 5, alpha = 0.05)

cat("TOST Results\n")
cat("============\n")
cat("Mean difference:", round(result$difference, 3), "\n")
cat("90% CI: [", round(result$ci[1], 3), ",", round(result$ci[2], 3), "]\n")
cat("Equivalence bounds: [-", result$delta, ", +", result$delta, "]\n")
cat("\nTest 1 (lower bound): t =", round(result$t1, 3), ", p =", round(result$p1, 4), "\n")
cat("Test 2 (upper bound): t =", round(result$t2, 3), ", p =", round(result$p2, 4), "\n")
cat("TOST p-value:", round(result$p_tost, 4), "\n")
cat("\nConclusion:", ifelse(result$equivalent, "Equivalence established", "Equivalence not established"), "\n")
TOST Results
============
Mean difference: 2.208 
90% CI: [ 0.254 , 4.162 ]
Equivalence bounds: [- 5 , + 5 ]

Test 1 (lower bound): t = 6.192 , p = 0 
Test 2 (upper bound): t = -2.398 , p = 0.0103 
TOST p-value: 0.0103 

Conclusion: Equivalence established

120.9 Comparison with Traditional t-test

Table 120.1: Comparison of Traditional t-test and TOST
Aspect Traditional t-test TOST
Null hypothesis No difference (\(\mu_1 = \mu_2\)) Not equivalent (\(|\mu_1 - \mu_2| \geq \Delta\))
Alternative hypothesis Difference exists (\(\mu_1 \neq \mu_2\)) Equivalent (\(|\mu_1 - \mu_2| < \Delta\))
Failure to reject Cannot claim difference Cannot claim equivalence
Question answered Are they different? Are they equivalent?

A non-significant t-test does not establish equivalence — it only indicates insufficient evidence for a difference. TOST provides positive evidence for equivalence.

120.10 Pros & Cons

120.10.1 Pros

TOST has the following advantages:

  • Provides positive evidence for equivalence rather than relying on failure to reject a null hypothesis of no difference.
  • Controls the Type I error rate at the nominal level \(\alpha\).
  • The confidence interval approach provides an intuitive visual assessment of equivalence.
  • Widely accepted in regulatory settings for bioequivalence studies.

120.10.2 Cons

TOST has the following disadvantages:

  • Requires pre-specification of the equivalence margin \(\Delta\), which may be difficult to justify.
  • Larger sample sizes are typically needed compared to traditional t-tests to achieve adequate power.
  • The choice of \(\Delta\) directly affects conclusions — a larger margin makes equivalence easier to establish.
  • Assumes the same distributional requirements as the underlying t-test (normality, independence).

120.11 Task

  1. Two laboratories measure the concentration of a chemical compound in the same samples. Lab A reports a mean of 45.2 (SD = 3.1, n = 15) and Lab B reports a mean of 44.8 (SD = 2.9, n = 15). Using an equivalence margin of \(\Delta = 2\), test whether the laboratories produce equivalent results.

  2. A company claims their new manufacturing process produces equivalent results to the old process. The equivalence margin is set at 10% of the old process mean. Design a study to test this claim with 80% power at \(\alpha = 0.05\).

  3. Explain why a non-significant two-sample t-test cannot be used to conclude equivalence. Illustrate with an example where the t-test is non-significant but TOST fails to establish equivalence.