The Smoothing Proximal Gradient Algorithm with Extrapolation for the Relaxation of $l_0$ Regularization Problem

Abstract

In this paper, we consider the exact continuous relaxation model of $l_0$ regularization problem which was given by Bian and Chen (SIAM J. Numer. Anal 58(1): 858-883, 2020) and propose a smoothing proximal gradient algorithm with extrapolation (SPGE) for this kind of problem. We show that any accumulation point of the sequence generated by SPGE algorithm is a lifted stationary point of the relaxation model for a fairly general choice of extrapolation parameter. Moreover, it is shown that the whole sequence generated by SPGE algorithm converges to a lifted stationary point of the relaxation model. The convergence rate of $o(\frac{1}{k^{1-\sigma}})(\sigma\in(0,1))$ with respect to squared proximal residual is established. Finally, we conduct three important numerical experiments to illustrate the faster efficiency of the SPGE algorithm compared with the smoothing proximal gradient(SPG) algorithm proposed by Bian and Chen (SIAM J. Numer. Anal 58(1):858-883, 2020).