Measurement and comparison of distributional shift with applications to ecology, economics, and image analysis

Abstract

The concentration of a distribution toward a lower bound is a conceptually simple property that closely relates to concepts of rarity and poverty, but that lacks a global descriptive statistic. We term this property 'shift' and define it as the distance of a central tendency from an upper bound, expressed as a proportion of a finite range. We derive a flexible, low complexity measure of shift and demonstrate its properties, its use with theoretical distributions, and its relation to skewness. We then use shift as the basis for a directional difference measure and as the basis for a formal distance metric that closely approximates the behavior of metrics having greater complexity (e.g., Wasserstein distance). Using simulated datasets and comparisons to system-specific measures, we demonstrate shift as a measure of species rarity and as a measure of poverty. We then apply our shift statistics to the analysis of image data. The shift statistics presented have a high degree of potential use across disciplines.