Strong log-concavity in probit regression

Abstract

We show that strong log-concavity emerges in probit regression likelihoods without ridge penalization (i.e. Gaussian priors), unlike for the logistic case. Specifically, we provide: (a) a characterization of strong log-concavity for fixed designs, similar to that for the existence of the maximum likelihood estimator (MLE) and (b) an analysis for Gaussian design, dependent on the proportionality $d/n = r\in [0, 1)$ between the sample size $n$ and the number of covariates $d$. In the latter case we show that, with high probability, provided $r$ is small enough, the resulting condition number is finite and, in the asymptotic regime $n, d\rightarrow \infty$, independent of $r$.