Wanted: standards for automatic reproducibility of computational experiments

Abstract

Those seeking to reproduce a computational experiment often need to manually look at the code to see how to build necessary libraries, configure parameters, find data, and invoke the experiment; it is not automatic. Automatic reproducibility is a more stringent goal, but working towards it would benefit the community. This work discusses a machine-readable language for specifying how to execute a computational experiment. We invite interested stakeholders to discuss this language at https://github.com/charmoniumQ/execution-description .