127  Kruskal-Wallis Test

The Kruskal-Wallis Test (Kruskal and Wallis 1952) is a non-parametric alternative to the One-way ANOVA (Chapter 126). It is used when the assumptions of the One-way ANOVA (normality, equal variances) are not satisfied, or when the data are measured on an ordinal scale. The test is based on ranks rather than raw values and extends the Mann-Whitney U test (Chapter 121) to three or more independent groups.

127.1 Hypotheses

The Kruskal-Wallis Test evaluates whether the distributions of all groups are identical:

\[ \begin{cases}\text{H}_0: F_1 = F_2 = \ldots = F_k \\\text{H}_A: \exists\; i \neq j: F_i \neq F_j\end{cases} \]

where \(F_i\) denotes the distribution function of group \(i\) and \(k\) is the number of independent groups.

Under the assumption that all groups have similar distributional shapes (differing only in location), the test effectively compares group medians. Without this assumption, a significant result indicates that at least two groups differ in their distributions, but the nature of the difference (location, spread, or shape) cannot be determined from the test alone.

The test statistic \(H\) is computed by

\[ H = \frac{12}{N(N+1)} \sum_{i=1}^{k} \frac{R_i^2}{n_i} - 3(N+1), \]

where \(R_i\) is the sum of ranks in group \(i\), \(n_i\) is the group size, and \(N = \\sum_i n_i\).

When ties are present, a tie-correction factor should be applied (R does this automatically).

Operationally this means:

1. Rank all observations from all groups together (ignoring group membership).
2. Compute the rank-sum \(R_i\) for each group.
3. Compare observed rank-sums to what would be expected if all groups had the same distribution.

The test statistic \(H\) approximately follows a Chi-squared distribution with \(k - 1\) degrees of freedom when sample sizes are sufficiently large.

127.2 Analysis based on p-values and confidence intervals

127.2.1 Software

The Kruskal-Wallis Test can be computed using the R module available in RFC under the “Hypotheses / Empirical Tests” menu item (select “Kruskal-Wallis Test” from the ANOVA type dropdown), or through “Hypotheses / Multivariate (pair-wise) Testing” (use the Boxplot tab with type “nonparametric”). An example using the multidesc app is available here (click on violin plot (between variance)).

127.2.2 Data & Parameters

The data should contain:

• A quantitative (or ordinal) response variable
• A categorical variable identifying the group membership
• Three or more independent groups

The data can be in wide format (one column per group) or long format (one column for the response, one column for the group label).

127.2.3 Output

Consider the same anorexia treatment example from Chapter 126 where we measure the post-therapy weight of patients across three treatments (“CBT”, “Control”, and “FT”). The results from the Kruskal-Wallis analysis are shown below.

Interactive Shiny app (click to load).

Open in new tab

The output reports the Kruskal-Wallis chi-squared statistic, group descriptive statistics, and (if the overall test is significant) post-hoc pairwise Wilcoxon rank-sum tests with Bonferroni correction. The plots show box plots for the raw values and ranks across groups.

To compute the same analysis on your local machine:

# Using the anorexia dataset
library(MASS)
data(anorexia)

# Kruskal-Wallis test
kruskal.test(Postwt ~ Treat, data = anorexia)

    Kruskal-Wallis rank sum test

data:  Postwt by Treat
Kruskal-Wallis chi-squared = 12.881, df = 2, p-value = 0.001596

The output reports the Kruskal-Wallis chi-squared statistic, the degrees of freedom (\(k - 1\), where \(k\) is the number of groups), and the p-value. If the p-value is smaller than the chosen type I error \(\alpha\), we reject the Null Hypothesis and conclude that at least two groups differ.

For reporting, include an effect size such as

\[ \epsilon^2 = \frac{H - k + 1}{N - k}. \]

127.2.3.1 Post-hoc: Dunn’s test

When the overall Kruskal-Wallis test is significant, Dunn’s test (Dunn 1964) with Bonferroni correction is used to determine which specific groups differ:

# Dunn's test for post-hoc pairwise comparisons
# Using the PMCMRplus package
if (requireNamespace("PMCMRplus", quietly = TRUE)) {
  library(PMCMRplus)
  kwAllPairsDunnTest(Postwt ~ Treat, data = anorexia, p.adjust.method = "bonferroni")
} else {
  # Alternative: pairwise Wilcoxon rank-sum tests with Bonferroni correction
  pairwise.wilcox.test(anorexia$Postwt, anorexia$Treat, p.adjust.method = "bonferroni")
}

     CBT    Cont  
Cont 0.0858 -     
FT   0.2853 0.0012

The Bonferroni correction inflates the p-values to account for multiple comparisons, controlling the family-wise type I error rate at the chosen \(\alpha\) level.

127.3 R code

To compute the Kruskal-Wallis Test on your local machine, the following script can be used in the R console:

# Example with the built-in iris dataset
# Testing whether Sepal.Width differs across Species
kruskal.test(Sepal.Width ~ Species, data = iris)

# Post-hoc: pairwise Wilcoxon rank-sum tests with Bonferroni correction
pairwise.wilcox.test(iris$Sepal.Width, iris$Species, p.adjust.method = "bonferroni")

    Kruskal-Wallis rank sum test

data:  Sepal.Width by Species
Kruskal-Wallis chi-squared = 63.571, df = 2, p-value = 1.569e-14


    Pairwise comparisons using Wilcoxon rank sum test with continuity correction 

data:  iris$Sepal.Width and iris$Species 

           setosa  versicolor
versicolor 6.4e-13 -         
virginica  2.1e-08 0.014     

P value adjustment method: bonferroni 

Note that the formula syntax response ~ group is used with kruskal.test(), which assumes the data is in long format. Alternatively, a list of numeric vectors can be passed directly.

127.4 Assumptions

The Kruskal-Wallis Test makes the following assumptions:

• Independent observations: The observations within and between groups must be independent (no paired or repeated measures).
• Ordinal or continuous outcome: The response variable must be measured on at least an ordinal scale.
• Similar distributional shape: For the test to be interpreted as a comparison of locations (medians), the distributions across groups should have similar shapes. This is the same assumption as for the Mann-Whitney U test (Chapter 121). Without this assumption, the test compares mean ranks rather than medians.

127.5 Alternatives

• One-way ANOVA (Chapter 126): When the assumptions of normality and equal variances are satisfied, the parametric ANOVA test has greater statistical power.
• Permutation tests: A flexible non-parametric alternative that does not require distributional assumptions but can be computationally intensive for large datasets.

Dunn, Olive Jean. 1964. “Multiple Comparisons Using Rank Sums.” Technometrics 6 (3): 241–52. https://doi.org/10.1080/00401706.1964.10490181.

Kruskal, William H., and W. Allen Wallis. 1952. “Use of Ranks in One-Criterion Variance Analysis.” Journal of the American Statistical Association 47 (260): 583–621. https://doi.org/10.1080/01621459.1952.10483441.