Intervenability as a Design Requirement for Autonomy and Oversight within Human-Centered AI

Abstract

Based on the literature and several practical examples of possible AI applica-tions, we outline the concept of intervenability. This new phenomenon is not covered by emergency shutdowns, workarounds, or the reconfiguration of automated systems. Intervenability instantiates the principles of control-lability, autonomy, oversight, and keeping humans in the loop in the context of AI. We provide a taxonomy that encompasses a range of possibilities for intervening activities and differentiates them regarding the mental effort of the users. This taxonomy extends the scope of interventions from real-time control of automated processes to AI-based discrete case-related decision-making. This is in accordance with human-centered AI, which seeks to combine human strengths with the usage of AI. We demonstrate how inter-venability can potentially contribute to the ongoing development of human capabilities on the one hand and to further technical improvement by recon-figuration of AI on the other. Exploring and collaboratively reflecting on the effects of interventions as an integral part of organizational practices is key to enabling this continuous improvement on both sides. Intervenability also provides further momentum for the design of an AI that can help realize in-terventions on its own and advance a smooth transition from intervention to reconfiguration of the AI.