140  Conditional Inference Trees

140.1 Definition

A conditional inference tree is a non-parametric classification and regression method that uses a statistical testing framework to select variables and determine split points. Unlike traditional decision trees (CART; Breiman et al. (1984)), conditional inference trees use permutation tests to evaluate the association between each predictor and the response variable, avoiding the variable selection bias that occurs with greedy algorithms.

The method was introduced by Hothorn, Hornik, and Zeileis (Hothorn, Hornik, and Zeileis (2006)) and is implemented in the party package in R via the ctree() function.

140.2 Algorithm

The algorithm proceeds as follows:

  1. Test for independence: For each predictor variable, test the null hypothesis of independence between the predictor and the response using permutation tests based on the conditional distribution of linear statistics.

  2. Variable selection: Select the predictor with the strongest association to the response (smallest p-value). If the smallest p-value is above the significance threshold, stop splitting.

  3. Split point selection: For the selected predictor, find the split point that maximizes a two-sample statistic.

  4. Recursion: Repeat the process for each child node until a stopping criterion is met.

140.3 Statistical Testing Framework

The key difference from traditional decision trees is the use of conditional inference. For each predictor \(X_j\), the algorithm computes a test statistic based on influence functions and a linear statistic:

\[ T_j = \text{vec}\left(\sum_{i=1}^n w_i g_j(X_{ji}) h(Y_i, (Y_1, \ldots, Y_n))^T\right) \]

where:

  • \(w_i\) are case weights
  • \(g_j\) is a transformation of the predictor (e.g., indicator functions for factors)
  • \(h\) is an influence function for the response

The p-value is computed using the permutation distribution of this statistic, conditional on the observed data.

140.4 Control Parameters

The ctree_control() function specifies the stopping criteria:

Table 140.1: Control parameters for conditional inference trees
Parameter Description Default
mincriterion 1 - p-value threshold for splitting 0.95
minsplit Minimum observations for attempting a split 20
minbucket Minimum observations in terminal nodes 7
maxdepth Maximum tree depth Inf

A higher mincriterion requires stronger evidence to split (more conservative tree). Setting mincriterion = 0.95 corresponds to a significance level of \(\alpha = 0.05\).

140.5 Comparison with CART

Table 140.2: Comparison of CART and Conditional Inference Trees
Aspect CART Conditional Inference Tree
Variable selection Greedy (maximizes impurity reduction) Statistical testing
Selection bias Favors variables with many splits Unbiased
Pruning Required (post-hoc) Built-in via significance testing
Split criterion Gini, entropy, or variance Permutation test p-values
Interpretability High High

140.6 R Module

140.6.1 Public website

Conditional Inference Trees are available on the public website:

140.6.2 RFC

The Conditional Inference Tree module is available in RFC under the menu “Models / Conditional Inference Tree”.

An interactive model-building application that includes conditional inference trees alongside other classification methods (logistic regression, naive Bayes, conditional forests) is available under “Models / Manual Model Building”. In its Tree tab, binary classification trees expose a classification threshold. That threshold converts predicted probabilities into predicted classes, so changing it changes the confusion matrix, sensitivity, specificity, and precision. This application therefore lets users compare model performance using ROC curves (Chapter 60) and confusion matrices (Chapter 59).

140.6.3 R Code

The following code demonstrates fitting a conditional inference tree for classification:

library(party)

# Create example data: customer churn prediction
set.seed(101)
n <- 400

# Predictors with clear effects on churn
months_subscribed <- sample(1:48, n, replace = TRUE)
monthly_charges <- runif(n, 20, 120)
support_calls <- rpois(n, lambda = 2)
contract_type <- factor(sample(c("Monthly", "Annual", "Two-Year"), n,
                               replace = TRUE, prob = c(0.5, 0.3, 0.2)))

# Churn probability: higher with short subscription, high charges, many support calls, monthly contract
log_odds <- -2 +
  ifelse(months_subscribed < 12, 1.5, ifelse(months_subscribed < 24, 0.5, -0.5)) +
  ifelse(monthly_charges > 80, 1.2, ifelse(monthly_charges > 50, 0.3, -0.5)) +
  0.3 * support_calls +
  ifelse(contract_type == "Monthly", 1, ifelse(contract_type == "Annual", 0, -1))

churn <- factor(ifelse(runif(n) < plogis(log_odds), "Yes", "No"))

df <- data.frame(months_subscribed = months_subscribed,
                 monthly_charges = monthly_charges,
                 support_calls = support_calls,
                 contract_type = contract_type,
                 churn = churn)

# Fit conditional inference tree
tree <- ctree(churn ~ months_subscribed + monthly_charges + support_calls + contract_type,
              data = df,
              controls = ctree_control(mincriterion = 0.95, minsplit = 20))
print(tree)

     Conditional inference tree with 7 terminal nodes

Response:  churn 
Inputs:  months_subscribed, monthly_charges, support_calls, contract_type 
Number of observations:  400 

1) months_subscribed <= 16; criterion = 1, statistic = 31.802
  2) monthly_charges <= 51.97214; criterion = 0.998, statistic = 11.857
    3)*  weights = 50 
  2) monthly_charges > 51.97214
    4)*  weights = 91 
1) months_subscribed > 16
  5) monthly_charges <= 97.80051; criterion = 1, statistic = 25.207
    6) contract_type == {Monthly}; criterion = 0.996, statistic = 14.079
      7) support_calls <= 1; criterion = 0.992, statistic = 9.546
        8)*  weights = 39 
      7) support_calls > 1
        9)*  weights = 56 
    6) contract_type == {Annual, Two-Year}
      10)*  weights = 109 
  5) monthly_charges > 97.80051
    11) contract_type == {Two-Year}; criterion = 0.999, statistic = 15.85
      12)*  weights = 10 
    11) contract_type == {Annual, Monthly}
      13)*  weights = 45

The tree can be visualized:

Code 
plot(tree)
Figure 140.1: Conditional inference tree for classification

140.7 Predictions

Predictions from a conditional inference tree include both class predictions and class probabilities:

# Class predictions
head(predict(tree))

# Class probabilities
probs <- do.call(rbind, treeresponse(tree))
colnames(probs) <- levels(df$churn)
head(probs)
[1] Yes No  No  No  No  No 
Levels: No Yes
            No       Yes
[1,] 0.2857143 0.7142857
[2,] 0.8990826 0.1009174
[3,] 0.8974359 0.1025641
[4,] 0.8990826 0.1009174
[5,] 0.8990826 0.1009174
[6,] 0.8990826 0.1009174

The predicted probabilities can be used for ROC analysis (Chapter 60) to evaluate classifier performance and determine optimal classification thresholds.

140.8 Example: Medical Diagnosis

A hospital wants to predict disease presence based on patient characteristics:

# Simulated medical data
set.seed(303)
n <- 600

# Patient characteristics
age <- sample(25:75, n, replace = TRUE)
bmi <- rnorm(n, mean = 27, sd = 6)
blood_pressure <- rnorm(n, mean = 125, sd = 20)
glucose <- rnorm(n, mean = 100, sd = 30)
smoking <- factor(sample(c("Never", "Former", "Current"), n,
                         replace = TRUE, prob = c(0.4, 0.35, 0.25)))

# Disease probability with strong, clear effects
log_odds <- -3 +
  ifelse(age > 55, 2.0, ifelse(age > 45, 0.8, -0.8)) +
  ifelse(bmi > 30, 1.5, ifelse(bmi > 27, 0.5, -0.5)) +
  ifelse(blood_pressure > 140, 1.5, ifelse(blood_pressure > 125, 0.5, -0.3)) +
  ifelse(glucose > 125, 1.2, ifelse(glucose > 100, 0.3, -0.3)) +
  ifelse(smoking == "Current", 1.8, ifelse(smoking == "Former", 0.6, -0.3))

disease <- factor(ifelse(runif(n) < plogis(log_odds), "Yes", "No"))

medical_data <- data.frame(age = age, bmi = bmi, blood_pressure = blood_pressure,
                           glucose = glucose, smoking = smoking,
                           disease = disease)

# Split into train and test
train_idx <- sample(1:n, 0.7 * n)
train <- medical_data[train_idx, ]
test <- medical_data[-train_idx, ]

# Fit tree
disease_tree <- ctree(disease ~ age + bmi + blood_pressure + glucose + smoking,
                      data = train,
                      controls = ctree_control(mincriterion = 0.95, minsplit = 20))

# Predictions on test set
pred_class <- predict(disease_tree, newdata = test)
pred_probs <- do.call(rbind, treeresponse(disease_tree, newdata = test))
colnames(pred_probs) <- levels(train$disease)

# Confusion matrix
table(Actual = test$disease, Predicted = pred_class)

# Accuracy
mean(pred_class == test$disease)
      Predicted
Actual  No Yes
   No  103   7
   Yes  38  32
[1] 0.75
Code 
plot(disease_tree)
Figure 140.2: Conditional inference tree for disease prediction

140.9 Classification Thresholds for Binary Trees

For a binary outcome, the tree does not only produce a class label. It first produces a probability for the positive class. The final class label is obtained by comparing that probability with a threshold.

Using a threshold of 0.50 means that the model predicts Yes only when the estimated probability of disease is at least 50%. Lowering the threshold makes positive predictions easier to trigger. That usually increases sensitivity but may reduce specificity and precision.

prob_disease <- pred_probs[, "Yes"]

class_at_threshold <- function(prob, threshold, positive = "Yes", negative = "No") {
  factor(ifelse(prob >= threshold, positive, negative), levels = c(negative, positive))
}

metrics_from_pred <- function(actual, predicted, positive = "Yes") {
  tab <- table(actual, predicted)
  TP <- tab[positive, positive]
  TN <- sum(tab) - sum(tab[positive, ]) - sum(tab[, positive]) + TP
  FN <- sum(tab[positive, ]) - TP
  FP <- sum(tab[, positive]) - TP
  data.frame(
    Accuracy = mean(predicted == actual),
    Sensitivity = TP / (TP + FN),
    Specificity = TN / (TN + FP),
    Precision = TP / (TP + FP)
  )
}

pred_050 <- class_at_threshold(prob_disease, 0.50)
pred_035 <- class_at_threshold(prob_disease, 0.35)

threshold_tab <- rbind(
  cbind(Threshold = 0.50, metrics_from_pred(test$disease, pred_050)),
  cbind(Threshold = 0.35, metrics_from_pred(test$disease, pred_035))
)

knitr::kable(
  transform(
    threshold_tab,
    Accuracy = round(Accuracy, 3),
    Sensitivity = round(Sensitivity, 3),
    Specificity = round(Specificity, 3),
    Precision = round(Precision, 3)
  ),
  caption = "Changing the threshold changes the binary tree's classification behavior"
)
Changing the threshold changes the binary tree’s classification behavior
Threshold Accuracy Sensitivity Specificity Precision
0.50 0.75 0.457 0.936 0.821
0.35 0.75 0.557 0.873 0.736

This is the same logic used in the Tree tab of Models / Manual Model Building: the fitted tree supplies class probabilities, and the threshold determines where the final yes/no boundary is drawn.

140.10 ROC Analysis Integration

The predicted probabilities from a conditional inference tree can be used to construct ROC curves (Chapter 60) for evaluating and comparing classifier performance:

# Get probability of disease (positive class)
# (already used above for threshold-based classification)
prob_disease <- pred_probs[, "Yes"]

# Manual ROC curve computation
thresholds <- sort(unique(c(0, prob_disease, 1)))
actual <- as.integer(test$disease == "Yes")

roc_points <- data.frame(threshold = numeric(), FPR = numeric(), TPR = numeric())

for (t in thresholds) {
  predicted <- as.integer(prob_disease >= t)
  TP <- sum(predicted == 1 & actual == 1)
  TN <- sum(predicted == 0 & actual == 0)
  FP <- sum(predicted == 1 & actual == 0)
  FN <- sum(predicted == 0 & actual == 1)
  TPR <- ifelse((TP + FN) > 0, TP / (TP + FN), 0)
  FPR <- ifelse((TN + FP) > 0, FP / (TN + FP), 0)
  roc_points <- rbind(roc_points, data.frame(threshold = t, FPR = FPR, TPR = TPR))
}

roc_points <- roc_points[order(roc_points$FPR), ]

# AUC computation (trapezoidal rule)
auc <- 0
for (i in 1:(nrow(roc_points) - 1)) {
  auc <- auc + (roc_points$FPR[i+1] - roc_points$FPR[i]) *
    (roc_points$TPR[i] + roc_points$TPR[i+1]) / 2
}

cat("AUC:", round(auc, 3), "\n")
AUC: 0.81
Code 
plot(roc_points$FPR, roc_points$TPR, type = "l", lwd = 2, col = "blue",
     xlab = "False Positive Rate (1 - Specificity)",
     ylab = "True Positive Rate (Sensitivity)",
     main = "ROC Curve - Conditional Inference Tree")
abline(0, 1, lty = 2, col = "gray")
legend("bottomright", legend = paste("AUC =", round(abs(auc), 3)),
       col = "blue", lwd = 2)
Figure 140.3: ROC curve for conditional inference tree classifier

140.11 Regression Trees

Conditional inference trees can also be used for regression (continuous response):

# Regression example
set.seed(42)
n <- 200
x1 <- runif(n, 0, 10)
x2 <- runif(n, 0, 10)
y <- 2 + 3 * (x1 > 5) + 2 * (x2 > 5) + rnorm(n, sd = 0.5)

reg_data <- data.frame(x1 = x1, x2 = x2, y = y)
reg_tree <- ctree(y ~ x1 + x2, data = reg_data)

# Predictions
pred <- predict(reg_tree)
cat("RMSE:", sqrt(mean((y - pred)^2)), "\n")
cat("R-squared:", cor(y, pred)^2, "\n")
RMSE: 0.4548386 
R-squared: 0.9394194
Code 
plot(reg_tree)
Figure 140.4: Conditional inference tree for regression

When the outcome is continuous, the tree plot itself is only the first diagnostic layer. The next question is whether the terminal nodes are equally reliable. Some leaves can have tight, well-behaved outcome distributions, whereas others can have broad spread or strong skewness even if their fitted mean looks usable. The follow-up chapter on leaf-wise distribution checks, Chapter 141, shows how to inspect that uncertainty inside the fitted terminal nodes rather than stopping at the tree diagram.

140.12 Variable Importance

While conditional inference trees do not have a built-in variable importance measure like random forests, the variables that appear higher in the tree (closer to the root) are generally more important. The number of times a variable is used for splitting across the tree can also indicate importance. If you need a model class that formalizes importance through an ensemble rather than a single diagram, the next chapter on Chapter 142 is the natural continuation.

140.13 Pros & Cons

140.13.1 Pros

Conditional inference trees have the following advantages:

  • Unbiased variable selection due to statistical testing framework.
  • No need for post-hoc pruning; tree size is controlled by significance threshold.
  • Handles mixed predictor types (numeric and categorical) naturally.
  • Provides interpretable decision rules.
  • Less prone to overfitting compared to CART when using appropriate mincriterion.

140.13.2 Cons

Conditional inference trees have the following disadvantages:

  • Computationally more intensive than CART due to permutation tests.
  • May produce smaller trees than CART, potentially underfitting in some cases.
  • The permutation test framework assumes exchangeability under the null hypothesis.
  • Variable importance is less straightforward than in ensemble methods.
  • A single fitted tree can be unstable if the training sample changes; Chapter 142 shows how ensembling is used to reduce that instability when prediction matters more than a single interpretable diagram.
  • Cannot extrapolate beyond the range of training data.

140.14 Task

  1. Using the iris dataset, fit a conditional inference tree to classify species based on sepal and petal measurements. Visualize the tree and interpret the decision rules.

  2. Compare the performance of a conditional inference tree with logistic regression (Chapter 136) on a binary classification problem. Use ROC curves and AUC (Section 60.4) to evaluate both models.

  3. Experiment with different values of mincriterion (0.90, 0.95, 0.99) and observe how tree complexity changes. Discuss the trade-off between tree size and prediction accuracy.

  4. Fit a regression tree to predict mpg in the mtcars dataset. Compare the predictions with those from a linear regression model (Chapter 134).