Explainable AI for computational pathology identifies model limitations and tissue biomarkers

Abstract

Deep learning models show promise in digital pathology, but their opaque decision-making processes limit trust and clinical adoption. To address this challenge, we present HIPPO, an explainable AI method for analyzing weakly-supervised multiple-instance learning (MIL) models that are widely used in whole slide image analysis. HIPPO constructs counterfactual whole slide images by systematically removing or adding selected tissue regions, providing a principled way to quantify how specific histologic areas influence model predictions under the MIL framework. This enables rigorous model interpretation, quantitative hypothesis testing, bias detection, and performance evaluation that extend beyond standard metrics. We demonstrate HIPPO across key clinical tasks, such as breast metastasis detection in axillary lymph nodes, prognostication in breast cancer and melanoma, and IDH mutation classification in gliomas. In metastasis detection, HIPPO uncovered critical model limitations undetectable by standard performance metrics or attention-based methods. For prognostic prediction, HIPPO outperformed attention by providing more nuanced insights into tissue elements influencing outcomes. In a proof-of-concept study, HIPPO facilitated hypothesis generation to identify melanoma patients who may benefit from immunotherapy. In IDH mutation classification, HIPPO more robustly identified the pathology regions responsible for false negatives compared to attention, suggesting its potential to outperform attention in explaining model decisions. In summary, HIPPO expands the explainable AI toolkit for computational pathology by enabling deeper insights into model behavior. This framework supports the trustworthy development, deployment, and regulation of weakly-supervised models in clinical and research settings, promoting their broader adoption in digital pathology.