The sign of the logistic regression coefficient

Abstract

Let Y be a binary random variable and X a scalar. Let $\hat\beta$ be the maximum likelihood estimate of the slope in a logistic regression of Y on X with intercept. Further let $\bar x_0$ and $\bar x_1$ be the average of sample x values for cases with y=0 and y=1, respectively. Then under a condition that rules out separable predictors, we show that sign($\hat\beta$) = sign($\bar x_1-\bar x_0$). More generally, if $x_i$ are vector valued then we show that $\hat\beta=0$ if and only if $\bar x_1=\bar x_0$. This holds for logistic regression and also for more general binary regressions with inverse link functions satisfying a log-concavity condition. Finally, when $\bar x_1\ne \bar x_0$ then the angle between $\hat\beta$ and $\bar x_1-\bar x_0$ is less than ninety degrees in binary regressions satisfying the log-concavity condition and the separation condition, when the design matrix has full rank.