Conditional independence testing through marginal permutation

Abstract

Marginal permutation is exact for testing marginal independence but generally invalid for conditional independence because it breaks the dependence between the exposure ($\bm X$) and covariates ($\bm Z$). In this paper, we show that empirical studentization can nevertheless make this deliberately misspecified resampling scheme asymptotically valid for important conditional independence tests based on scores. The resulting procedures are novel in the sense that they are valid in finite samples when the exposure is marginally independent of the outcome ($\bm Y$) and covariates, and asymptotically valid under the broader conditional independence null, $\bm X\indep \bm Y\mid \bm Z$, subject to method-specific conditions. For generalized linear model scores, separately studentizing the observed and permuted scores yields validity under a correctly specified null mean despite \emph{arbitrary} conditional variance misspecification. Moreover, the permutation calibration also applies to tests based on generalized and projected covariance measures. Their validity accommodates machine learning nuisance estimates under a condition on the stability of refitting after permutation, for which we provide sufficient conditions for common regression learners. Permutation calibration preserves first-order local power while better capturing the finite-sample discreteness and skewness induced by sparse observations. Simulations and single-cell CRISPR screen analysis show substantial calibration gains with our methods when effective sample size is small.