A self-governing, self-regulating system for assessing scientific predictive power

Abstract

I propose a method for tracking and assessing scientific progress using a prediction consensus algorithm designed for the purpose. The protocol obviates the need for centralized referees to generate scientific questions, gather predictions, and assess the accuracy or success of those predictions. It relies instead on crowd wisdom and a system of checks and balances for all tasks. It is intended to take the form of a web-based, searchable database. I describe a prototype implementation that I call Ex Quaerum. The main purpose of the present document is to motivate the project, to explain its underlying philosophy, to explain the details of the consensus protocol on which it is based, to describe plans for its future development, and ultimately to attract additional collaborators.