Understanding Biomedical Machine Learning Models

Abstract

As complex, black box models have increasingly come to predominate the algorithms used in state-of-the-art machine learning pipelines, the need to explain and understand the predictions made by these algorithms has grown correspondingly. Feature attribution methods are one popular approach to explain these black box models, but are limited in their expressive capacity. We therefore propose three approaches to go beyond the shortcomings of existing feature attribution methods. The first, EXPRESS, demonstrates how the stability and quality of feature attributions for models of gene expression data increase when these models are ensembled. The second, Integrated Hessians, efficiently explains the interactions between pairs of features for neural network models, which we show has general applications even beyond biological and medical models. In a third approach, we apply generative adversarial networks and saliency maps to identify the underlying reasons for poor generalizability of radiographic COVID-19 detection models. Furthermore, while the utility of feature attribution methods for helping humans understand what models have learned is well-known, their utility for helping humans express their own desiderata in machine-interpretable language is under-appreciated. We develop a feature attribution method that is designed for use during model training, and demonstrate how it can be used to incorporate gene interaction networks as a constraint on predictive models with gene expression features. Finally, we show how to enforce more abstract model constraints using adversarial training in the context of radiographic pneumonia classification.