Guardrails versus Gatekeepers: Understanding Product Managers' Ethical Decision-Making in Generative AI

Abstract

What is the role of product managers in the responsible use of generative AI (genAI) in products and everyday work -- and what enables or constrains their ability to take action? Past literature has examined the ways in which organizational policies can become decoupled from practices when incentives for responsible action are misaligned or impeded by profit motives. While the role of engineers and professional ethicists in the context of AI has been examined in detail, the role of product managers -- who are frequently portrayed as "gatekeepers" or critical decision-makers in product teams -- remains unclear. In this paper, we examine what organizational conditions promote responsible use of genAI by product managers by drawing on twenty-five interviews and a global survey of over three hundred respondents in product management-related roles. We find that uncertainty around responsible AI and a sense of diffused responsibility constrain ethical action, while leadership commitment and organizational principles enable ethical action -- making some responsible practices up to fourteen times more likely. Further, we find two sets of actions product managers take to "recouple" ethical commitments and practices. The first includes low-resource, individual actions product managers can implement without explicit organizational incentives. The second includes high-resource, collective actions that require organizational incentives. Our research suggests recoupling ethical policies and practices at the level of product teams requires institutional buy-in and higher level leadership commitment. Nevertheless, we show that individual actors are able to exhibit agency through some meaningful, low resource actions, even in the absence of organizational incentives, though this alone is insufficient to operationalize responsible AI at scale.