Interpreting Neural Network Judgments via Minimal, Stable, and Symbolic Corrections.

The paper describes a new algorithm to generate minimal, stable, and symboliccorrections to an input that will cause a neural network with ReLU neurons tochange its output. We argue that such a correction is a useful way to providefeedback to a user when the neural network produces an output that is differentfrom a desired output. Our algorithm generates such a correction by solving aseries of linear constraint satisfaction problems. The technique is evaluatedon a neural network that has been trained to predict whether an applicant willpay a mortgage.