Lord's 'paradox' explained: the 50-year warning on the use of 'change scores' in observational data

Abstract

In 1967, Frederick Lord posed a conundrum that has confused scientists for over half a century. Subsequently named Lord's 'paradox', the puzzle centres on the observation that two different approaches to estimating the effect of an exposure on the 'change' in an outcome can produce radically different results. Approach 1 involves comparing the mean 'change score' between exposure groups and Approach 2 involves comparing the follow-up outcome between exposure groups conditional on the baseline outcome. Resolving this puzzle starts with recognising the three reasons that a variable may change value: (A) 'endogenous change', which represents autocorrelation from baseline, (B) 'random change', which represents change from transient random processes, and (C) 'exogenous change', which represents all non-endogenous, non-random change and contains all change that is potentially modifiable by other baseline variables. In observational data, neither Approach 1 nor Approach 2 can reliably estimate the causal effect of an exposure on 'exogenous change' in an outcome. Approach 1 is susceptible to diluted or opposite-sign estimates whenever the exposure causes, or is caused by, the baseline outcome. Approach 2 is susceptible to inflated estimates due to measurement error in the baseline outcome and time-varying confounding bias when the baseline outcome is a mediator. The measurement error can be reduced with multiple measures of the baseline outcome, and the time-varying confounding can be reduced using g- methods. Lord's 'paradox' offers several enduring lessons for observational data science including the importance of a well-defined research question and the problems with analysing change scores in observational data.