Metrics & Evaluation Interactive

ROC-AUC

Measure classifier performance across all thresholds. AUC = probability that model ranks a random positive higher than a random negative.

📊 ROC Curve Basics

What It Measures

TPR (True Positive Rate) = TP / (TP + FN)
a.k.a. Sensitivity, Recall
FPR (False Positive Rate) = FP / (FP + TN)
a.k.a. 1 - Specificity

AUC Interpretation

• 1.0 = Perfect classifier
• 0.9+ = Excellent
• 0.7-0.9 = Good to Fair
• 0.5 = Random guessing

Model Quality

0 3

0.1 0.9

Higher separation = better model can distinguish classes

📊 AUC Score

0.931

Excellent

At Threshold 0.50

TPR (Recall) 82.1%

FPR 11.4%

Precision 86.7%

Accuracy 85.5%

ROC Curve

Curve above diagonal = better than random. Area under curve = AUC.

Confusion Matrix

Pred: Pos

Pred: Neg

Actual: Pos

Actual: Neg

🎯 When to Use ROC-AUC

✓ Good For

• Comparing models across all thresholds
• Balanced class problems
• When threshold is flexible
• Ranking quality (who's more likely?)

✗ Limitations

• Imbalanced classes (use PR-AUC instead)
• When you need specific threshold
• Doesn't measure calibration
• Can mislead with rare events

R Code Equivalent

# Calculate ROC-AUC
library(pROC)

# From predictions and actual
roc_obj <- roc(actual, predicted_prob)
auc_value <- auc(roc_obj)
cat(sprintf("AUC: %.3f\n", auc_value))

# Plot ROC curve
plot(roc_obj, main = "ROC Curve", 
     col = "#f5c542", lwd = 2)
abline(a = 0, b = 1, col = "gray", lty = 2)

# Confusion matrix at threshold
threshold <- 0.5
predicted_class <- ifelse(predicted_prob >= threshold, 1, 0)
table(Actual = actual, Predicted = predicted_class)

# Calculate metrics
library(caret)
confusionMatrix(factor(predicted_class), factor(actual))

✅ Key Takeaways

• ROC plots TPR vs FPR at all thresholds
• AUC = area under curve (higher = better)
• 0.5 = random, 1.0 = perfect

• Threshold-independent metric
• Use PR-AUC for imbalanced classes
• Measures ranking, not calibration