Reprowd: Crowdsourced Data Processing Made Reproducible

Abstract

Crowdsourcing is a multidisciplinary research area including disciplines like artificial intelligence, human-computer interaction, database, and social science. To facilitate cooperation across disciplines, reproducibility is a crucial factor, but unfortunately, it has not gotten enough attention in the HCOMP community. In this paper, we present Reprowd, a system aiming to make it easy to reproduce crowdsourced data processing research. We have open sourced Reprowd at http://sfu-db.github.io/reprowd/.