Data Cards: Purposeful and Transparent Dataset Documentation for Responsible AI
Mahima Pushkarna, Andrew Zaldivar, Oddur Kjartansson
Abstract
As research and industry moves towards large-scale models capable of numerous downstream tasks, the complexity of understanding multi-modal datasets that give nuance to models rapidly increases. A clear and thorough understanding of a dataset's origins, development, intent, ethical considerations and evolution becomes a necessary step for the responsible and informed deployment of models, especially those in people-facing contexts and high-risk domains. However, the burden of this understanding often falls on the intelligibility, conciseness, and comprehensiveness of the documentation. It requires consistency and comparability across the documentation of all datasets involved, and as such documentation must be treated as a user-centric product in and of itself. In this paper, we propose Data Cards for fostering transparent, purposeful and human-centered documentation of datasets within the practical contexts of industry and research. Data Cards are structured summaries of essential facts about various aspects of ML datasets needed by stakeholders across a dataset's lifecycle for responsible AI development. These summaries provide explanations of processes and rationales that shape the data and consequently the models, such as upstream sources, data collection and annotation methods; training and evaluation methods, intended use; or decisions affecting model performance. We also present frameworks that ground Data Cards in real-world utility and human-centricity. Using two case studies, we report on desirable characteristics that support adoption across domains, organizational structures, and audience groups. Finally, we present lessons learned from deploying over 20 Data Cards.
Create a lesson
Related papers
Souffleur: A Glanceable Display for Mobile Voice Conversations
Anton Wolter, William Benjamin, Niklas Elmqvist
ControlAnything: Generative UI and On-Demand Device Integration for Mixed-Reality Hand Interaction by a Coding Agent
Bektur Ryskeldiev
IntentWeaver: Sustaining Coherence across Evolving Intents in Human-AI Co-Writing
Yuanhao Zhang, Xinru Cheng, Baiqiao Zhang et al.
Reflective Nature Experiences in Virtual Reality
Shiva Ghasemi, Maddie Oehler, Annie Xenakis et al.
Tasteply: Externalizing Stable and Situational Preference as Manipulable Spatial Objects
DaEun Choi, Daniel Lee
VocabulARy 2.0: A Generative AR System Enabling Personalised 3D Mnemonics for Vocabulary Learning in Context
Karolina Trajkovska, Matjaž Kljun, Maheshya Weerasinghe et al.