install.packages("TOSTER")112 Two One-Sided Tests (TOST) for Equivalence
112.1 Definition
The Two One-Sided Tests (TOST) procedure (Schuirmann 1987) is a method for testing whether two means are equivalent within a pre-specified margin. Unlike traditional hypothesis testing which tests for a difference, equivalence testing tests whether any difference is small enough to be considered negligible.
In traditional hypothesis testing (Chapter 96), the null hypothesis states that there is no difference between groups, and we reject this null when evidence suggests a difference exists. In equivalence testing, the logic is reversed: the null hypothesis states that the groups are not equivalent, and we reject this null when evidence suggests the groups are equivalent.
112.2 The Equivalence Margin
The equivalence margin \(\Delta\) (delta) defines the largest difference that is considered practically unimportant. If the true difference between means lies within the interval \([-\Delta, +\Delta]\), the two groups are considered equivalent.
The choice of \(\Delta\) must be made before data collection and should be based on:
- Domain knowledge about what constitutes a meaningful difference
- Regulatory requirements (e.g., bioequivalence studies typically use \(\Delta = 0.2\) or 20%)
- Clinical or practical significance thresholds
112.3 Hypotheses
The TOST procedure decomposes the equivalence hypothesis into two one-sided hypotheses:
Test 1 (Lower bound):
\[ H_{01}: \mu_1 - \mu_2 \leq -\Delta \quad \text{vs.} \quad H_{a1}: \mu_1 - \mu_2 > -\Delta \]
Test 2 (Upper bound):
\[ H_{02}: \mu_1 - \mu_2 \geq +\Delta \quad \text{vs.} \quad H_{a2}: \mu_1 - \mu_2 < +\Delta \]
Equivalence is concluded if both null hypotheses are rejected at significance level \(\alpha\). This is equivalent to checking whether the \((1 - 2\alpha)\) confidence interval for the difference falls entirely within \([-\Delta, +\Delta]\).
112.4 Test Statistic
For each one-sided test, the test statistic follows the same form as the two-sample t-test (Chapter 110):
\[ t_1 = \frac{(\bar{x}_1 - \bar{x}_2) - (-\Delta)}{SE} = \frac{(\bar{x}_1 - \bar{x}_2) + \Delta}{SE} \]
\[ t_2 = \frac{(\bar{x}_1 - \bar{x}_2) - (+\Delta)}{SE} = \frac{(\bar{x}_1 - \bar{x}_2) - \Delta}{SE} \]
where \(SE\) is the standard error of the difference between means. For independent samples with equal variances:
\[ SE = s_p \sqrt{\frac{1}{n_1} + \frac{1}{n_2}} \]
where \(s_p\) is the pooled standard deviation.
112.5 Decision Rule
Equivalence is established if:
- \(t_1 > t_{\alpha, df}\) (reject \(H_{01}\)), AND
- \(t_2 < -t_{\alpha, df}\) (reject \(H_{02}\))
Alternatively, using p-values: equivalence is established if both \(p_1 < \alpha\) and \(p_2 < \alpha\).
The overall p-value for the TOST procedure is the larger of the two one-sided p-values:
\[ p_{TOST} = \max(p_1, p_2) \]
112.6 Confidence Interval Approach
An equivalent approach uses the \((1 - 2\alpha) \times 100\%\) confidence interval for the difference \(\mu_1 - \mu_2\):
\[ (\bar{x}_1 - \bar{x}_2) \pm t_{\alpha, df} \times SE \]
If this confidence interval lies entirely within \([-\Delta, +\Delta]\), equivalence is concluded at significance level \(\alpha\).
112.7 Example
A pharmaceutical company wants to demonstrate that a generic drug is bioequivalent to a brand-name drug. The equivalence margin is set at \(\Delta = 5\) units. Two groups of patients receive either the generic (\(n_1 = 24\)) or brand-name (\(n_2 = 24\)) drug, and a pharmacokinetic measure is recorded.
Results:
- Generic: \(\bar{x}_1 = 102.3\), \(s_1 = 4.3\)
- Brand: \(\bar{x}_2 = 100.1\), \(s_2 = 3.8\)
- Difference: \(\bar{x}_1 - \bar{x}_2 = 2.2\)
The 90% confidence interval for the difference (using \(\alpha = 0.05\) for each one-sided test) is \([0.3, 4.2]\).
Since the entire 90% confidence interval lies within \([-5, 5]\), equivalence can be established at \(\alpha = 0.05\). The generic drug is shown to be bioequivalent to the brand-name drug based on this study.
112.8 R Module
112.8.1 Public website
The TOST procedure is available on the public website:
112.8.2 RFC
The TOST module is available in RFC under the menu “Hypothesis Testing / Equivalence Testing (TOST)”.
112.8.3 R Code
The following code demonstrates the TOST procedure using the TOSTER package:
library(TOSTER)
# Example data
generic <- c(98, 105, 102, 99, 108, 101, 95, 110, 103, 97,
106, 100, 104, 99, 107, 102, 96, 109, 101, 98,
105, 103, 100, 108)
brand <- c(96, 102, 100, 98, 105, 99, 94, 107, 101, 95,
103, 98, 102, 97, 104, 100, 94, 106, 99, 96,
103, 101, 98, 105)
# TOST procedure with equivalence margin of 5
result <- tsum_TOST(
m1 = mean(generic),
sd1 = sd(generic),
n1 = length(generic),
m2 = mean(brand),
sd2 = sd(brand),
n2 = length(brand),
eqb = 5, # equivalence bound (symmetric)
alpha = 0.05
)
print(result)
Welch Two Sample t-test
The equivalence test was significant, t(45.39) = -2.398, p = 1.03e-02
The null hypothesis test was non-significant, t(45.39) = 1.897, p = 6.42e-02
NHST: don't reject null significance hypothesis that the effect is equal to zero
TOST: reject null equivalence hypothesis
TOST Results
t df p.value
t-test 1.897 45.39 0.064
TOST Lower 6.192 45.39 < 0.001
TOST Upper -2.398 45.39 0.01
Effect Sizes
Estimate SE C.I. Conf. Level
Raw 2.2083 1.1640 [0.2538, 4.1629] 0.9
Hedges's g(av) 0.5385 0.3003 [0.0596, 1.0117] 0.9
Note: SMD confidence intervals are an approximation. See vignette("SMD_calcs").For a manual implementation without external packages:
# Manual TOST implementation
tost_test <- function(x1, x2, delta, alpha = 0.05) {
n1 <- length(x1)
n2 <- length(x2)
m1 <- mean(x1)
m2 <- mean(x2)
# Pooled standard deviation
sp <- sqrt(((n1 - 1) * var(x1) + (n2 - 1) * var(x2)) / (n1 + n2 - 2))
se <- sp * sqrt(1/n1 + 1/n2)
df <- n1 + n2 - 2
# Difference
diff <- m1 - m2
# Two one-sided tests
t1 <- (diff + delta) / se # Test against lower bound
t2 <- (diff - delta) / se # Test against upper bound
p1 <- pt(t1, df, lower.tail = FALSE) # Upper tail for t1
p2 <- pt(t2, df, lower.tail = TRUE) # Lower tail for t2
# Confidence interval (90% for alpha = 0.05)
t_crit <- qt(1 - alpha, df)
ci_lower <- diff - t_crit * se
ci_upper <- diff + t_crit * se
# Decision
equivalent <- (p1 < alpha) & (p2 < alpha)
list(
difference = diff,
se = se,
t1 = t1, p1 = p1,
t2 = t2, p2 = p2,
p_tost = max(p1, p2),
ci = c(ci_lower, ci_upper),
delta = delta,
equivalent = equivalent
)
}
# Apply to example data
result <- tost_test(generic, brand, delta = 5, alpha = 0.05)
cat("TOST Results\n")
cat("============\n")
cat("Mean difference:", round(result$difference, 3), "\n")
cat("90% CI: [", round(result$ci[1], 3), ",", round(result$ci[2], 3), "]\n")
cat("Equivalence bounds: [-", result$delta, ", +", result$delta, "]\n")
cat("\nTest 1 (lower bound): t =", round(result$t1, 3), ", p =", round(result$p1, 4), "\n")
cat("Test 2 (upper bound): t =", round(result$t2, 3), ", p =", round(result$p2, 4), "\n")
cat("TOST p-value:", round(result$p_tost, 4), "\n")
cat("\nConclusion:", ifelse(result$equivalent, "Equivalence established", "Equivalence not established"), "\n")TOST Results
============
Mean difference: 2.208
90% CI: [ 0.254 , 4.162 ]
Equivalence bounds: [- 5 , + 5 ]
Test 1 (lower bound): t = 6.192 , p = 0
Test 2 (upper bound): t = -2.398 , p = 0.0103
TOST p-value: 0.0103
Conclusion: Equivalence established 112.9 Comparison with Traditional t-test
| Aspect | Traditional t-test | TOST |
|---|---|---|
| Null hypothesis | No difference (\(\mu_1 = \mu_2\)) | Not equivalent (\(|\mu_1 - \mu_2| \geq \Delta\)) |
| Alternative hypothesis | Difference exists (\(\mu_1 \neq \mu_2\)) | Equivalent (\(|\mu_1 - \mu_2| < \Delta\)) |
| Failure to reject | Cannot claim difference | Cannot claim equivalence |
| Question answered | Are they different? | Are they equivalent? |
A non-significant t-test does not establish equivalence — it only indicates insufficient evidence for a difference. TOST provides positive evidence for equivalence.
112.10 Pros & Cons
112.10.1 Pros
TOST has the following advantages:
- Provides positive evidence for equivalence rather than relying on failure to reject a null hypothesis of no difference.
- Controls the Type I error rate at the nominal level \(\alpha\).
- The confidence interval approach provides an intuitive visual assessment of equivalence.
- Widely accepted in regulatory settings for bioequivalence studies.
112.10.2 Cons
TOST has the following disadvantages:
- Requires pre-specification of the equivalence margin \(\Delta\), which may be difficult to justify.
- Larger sample sizes are typically needed compared to traditional t-tests to achieve adequate power.
- The choice of \(\Delta\) directly affects conclusions — a larger margin makes equivalence easier to establish.
- Assumes the same distributional requirements as the underlying t-test (normality, independence).
112.11 Task
Two laboratories measure the concentration of a chemical compound in the same samples. Lab A reports a mean of 45.2 (SD = 3.1, n = 15) and Lab B reports a mean of 44.8 (SD = 2.9, n = 15). Using an equivalence margin of \(\Delta = 2\), test whether the laboratories produce equivalent results.
A company claims their new manufacturing process produces equivalent results to the old process. The equivalence margin is set at 10% of the old process mean. Design a study to test this claim with 80% power at \(\alpha = 0.05\).
Explain why a non-significant two-sample t-test cannot be used to conclude equivalence. Illustrate with an example where the t-test is non-significant but TOST fails to establish equivalence.