Sparse higher order partial least squares for simultaneous variable selection, dimension reduction, and tensor denoising

Abstract

Partial Least Squares (PLS) regression emerged as an alternative to ordinary least squares for addressing multicollinearity in a wide range of scientific applications. As multidimensional tensor data is becoming more widespread, tensor adaptations of PLS have been developed. In this paper, we first establish the statistical behavior of Higher Order PLS (HOPLS) of Zhao et al. (2012), by showing that the consistency of the HOPLS estimator cannot be guaranteed as the tensor dimensions and the number of features increase faster than the sample size. To tackle this issue, we propose Sparse Higher Order Partial Least Squares (SHOPS) regression and an accompanying algorithm. SHOPS simultaneously accommodates variable selection, dimension reduction, and tensor response denoising. We further establish the asymptotic results of the SHOPS algorithm under a high-dimensional regime. The results also complete the unknown theoretic properties of SPLS algorithm (Chun and Kele\c{s}, 2010). We verify these findings through comprehensive simulation experiments, and application to an emerging high-dimensional biological data analysis.