Forecasting skill of a crowd-prediction platform: A comparison of exchange rate forecasts

Abstract

Open online crowd-prediction platforms are increasingly used to forecast trends and complex events. Despite the large body of research on crowd-prediction and forecasting tournaments, online crowd-prediction platforms have never been directly compared to other forecasting methods. In this analysis, exchange rate crowd-predictions made on Metaculus are compared to predictions made by the random-walk, a statistical model considered extremely hard-to-beat. The random-walk provides less erroneous forecasts, but the crowd-prediction does very well. By using the random-walk as a benchmark, this analysis provides a rare glimpse into the forecasting skill displayed on open online crowd-prediction platforms.