Separable Bregman Framework for Sparsity Constrained Nonlinear Optimization

Abstract

This paper considers the minimization of a continuously differentiable function over a cardinality constraint. We focus on smooth and relatively smooth functions. These smoothness criteria result in new descent lemmas. Based on the new descent lemmas, novel optimality conditions and algorithms are developed, which extend the previously proposed hard-thresholding algorithms. We give a theoretical analysis of these algorithms and extend previous results on properties of iterative hard thresholding-like algorithms. In particular, we focus on the weighted $\ell_2$ norm, which requires efficient solution of convex subproblems. We apply our algorithms to compressed sensing problems to demonstrate the theoretical findings and the enhancements achieved through the proposed framework.