A Case for Humans-in-the-Loop: Decisions in the Presence of Misestimated Algorithmic Scores

The increased use of machine learning to assist with decision-making in high-stakes domains has been met with both enthusiasm and concern. One source of ongoing debate is the effect and value of decision makers' discretionary power to override algorithmic recommendations. In this paper, we study the adoption of an algorithmic tool used to help with decisions in child maltreatment hotline screenings. By taking advantage of an implementation glitch, we investigate corrective overrides: whether decision makers are more likely to override algorithmic recommendations when the tool misestimates the risk score shown to call workers. We find that, after the deployment of the tool, decisions became better aligned with algorithmic assessments, but human adherence to the tool's recommendation was less likely when the displayed score was misestimated as a result of the glitch. Then, analyzing the effect of adoption and overrides on racial and socioeconomic disparities, we find that the deployment of the tool did not affect disparities with respect to the pre-deployment period. We also observe that the disparities resulting from algorithmic-informed decisions were substantially smaller than those associated with the algorithm in isolation. Together, these results make a case for the value of humans in-the-loop, showing that in high-stakes contexts, human discretionary power can mitigate the risks of algorithmic errors and reduce disparities.