Selective prediction‐set models with coverage rate guarantees

The current approach to using machine learning (ML) algorithms in healthcare is to either require clinician oversight for every use case or use their predictions without any human oversight. We explore a middle ground that lets ML algorithms abstain from making a prediction to simultaneously improve their reliability and reduce the burden placed on human experts. To this end, we present a general penalized loss minimization framework for training selective prediction‐set (SPS) models, which choose to either output a prediction set or abstain. The resulting models abstain when the outcome is difficult to predict accurately, such as on subjects who are too different from the training data, and achieve higher accuracy on those they do give predictions for. We then introduce a model‐agnostic, statistical inference procedure for the coverage rate of an SPS model that ensembles individual models trained using K‐fold cross‐validation. We find that SPS ensembles attain prediction‐set coverage rates closer to the nominal level and have narrower confidence intervals for its marginal coverage rate. We apply our method to train neural networks that abstain more for out‐of‐sample images on the MNIST digit prediction task and achieve higher predictive accuracy for ICU patients compared to existing approaches.