How to quantify direct correlations between variables

Abstract

A crucial question throughout statistics is whether an observed correlation between two variables is a direct correlation or only an indirect one mediated by a confounder. We organize the existing nonlinear measures of direct correlation into two families, each with a systematic construction: (i) removing the direct correlation from the joint distribution and quantifying the resulting distributional shift, and (ii) intervening on one variable via do-calculus and quantifying the response of the other. For every Kullback-Leibler-based measure in either family we propose a Jensen-Shannon-based regularized analogue; the regularized measures take values in $[0,1]$, satisfy the metric property, and are free of the singularities of the Kullback-Leibler divergence. We analyze the achievable upper bound of each regularized measure under the observed marginals, and derive the maximal value each measure can attain when only the alphabet sizes of the variables are fixed; the maxima admit closed forms built on a single binary-entropy function. The measures are compared on a decision-making model and on three public datasets (Titanic survival, UCI Adult income, and the 1973 Berkeley graduate admissions), with bootstrap confidence intervals for every reported value.