Convex Reparameterization and Self-Concordant Algorithms for Multivariate Regression with Covariance Estimation

Abstract

Building on a reparameterization for multivariate linear regression that yields a jointly convex penalized likelihood in the reparameterized regression coefficient matrix and the precision matrix, we show that the resulting scaled Gaussian loss is standard self-concordant. This places the joint estimation problem within composite self-concordant optimization and leads to two algorithms: a proximal gradient method and a damped proximal Newton method. In simulations, we evaluate algorithmic robustness, iterations to convergence, and elapsed time. In a protein expression application, compared with the classical-parameterization formulation, the proposed convex formulation attains similar mean squared prediction error and can be substantially faster when the fitted precision matrix is dense.