What do I make of your Latinorum? Sensitivity auditing of mathematical modelling

Abstract

In their book Uncertainty and Quality of Science for Policy (1990), Funtowicz and Ravetz argued the need to extend traditional methods and techniques of quality assurance in policy-related science. Since then, these ideas have been operationalized further and applied. Particularly relevant have been the recourse to extended peer review, to be intended as internal, across disciplines, as well as external, between practitioners and stakeholders, and the use of a new approach to qualify quantities: NUSAP (Numeral, Unit, Spread, Assessment, Pedigree). Here we describe how sensitivity analysis, mandated by existing guidelines as a good practice to use in conjunction to mathematical modelling, needs to be transformed and adapted to ensure quality in the treatment of uncertainty of science for policy. We thus provide seven rules to extend the use of sensitivity analysis (or how to apportion uncertainty in model based inference among input factors) in a process of sensitivity auditing of models used in a policy context. Each rule will be illustrated by examples.