Validation of machine learning based scenario generators

Abstract

Machine learning (ML) methods are becoming increasingly important in the design economic scenario generators for internal models. Validation of data-driven models differs from classical theory-based models. We discuss two novel aspects of such a validation: first, checking dependencies between risk factors and second, detecting unwanted memorization effects. The first task becomes necessary since in ML-based methods dependencies are no longer derived from a financial-mathematical theory but are driven by data. The need for the latter task arises since it cannot be ruled out that ML-based models merely reproduce the empirical data rather than generating new scenarios. To address the first issue, we propose to use an existing test from the literature. For the second issue, we introduce and discuss a novel memorization ratio. Numerical experiments based on real market data are included and an autoencoder-based scenario generator is validated with these two methods.