Resume Screening, Fast and Slow: (Biased) AI Recommendations' Influence on Human Decision Making

Abstract

AI is increasingly being used collaboratively with people to make decisions in high-stakes domains, but this new paradigm is still not well-understood in many respects -- particularly regarding how AI that replicates human social biases influences people's decision making processes and how that can influence outcomes. In this study, we analyzed the time people spend viewing candidate resumes from an experiment investigating biased AI resume screening to evaluate decision-making fairness and cognitive processes underlying human-AI collaboration. We found that spending more time viewing resumes corresponds to candidates' selection chance increasing by 3-4% if they are not recommended, and people may spend up to 55.6% longer viewing resumes when no AI recommendations are given. Furthermore, people who completed an implicit association test (IAT) before resume screening were significantly more likely to evaluate candidates of different races for the same amount of time, and their IAT scores were also predictive of the time spent in human-AI collaboration. These results demonstrate how people's decision-making processes can be insufficient for overseeing AI in high-stakes domains.