Small increases in agent-based model complexity can result in large increases in required calibration data

Abstract

Agent-based models (ABMs) are widely used to model coupled natural-human systems. Descriptive models require careful calibration with observed data. However, ABMs are often not calibrated in a statistical sense. Here we examine the impact of data record structure on the calibration of an ABM for housing abandonment in the presence of flood risk. Using a perfect model experiment, we examine the impact of data record structures on (i) model calibration and (ii) the ability to distinguish a model with inter-agent interactions from one without. We show how limited data sets may not constrain a model with just four parameters. This indicates that many ABMs may require informative prior distributions to be descriptive. We also illustrate how spatially-aggregated data can be insufficient to identify the correct model structure. This emphasizes the need for utilizing independent lines of evidence to select sound and informative priors.