Convolution based smooth approximations to the absolute value function with application to non-smooth regularization

Abstract

We present new convolution based smooth approximations to the absolute value function and apply them to construct gradient based algorithms such as the nonlinear conjugate gradient scheme to obtain sparse, regularized solutions of linear systems $Ax = b$, a problem often tackled via iterative algorithms which attack the corresponding non-smooth minimization problem directly. In contrast, the approximations we propose allow us to replace the generalized non-smooth sparsity inducing functional by a smooth approximation of which we can readily compute gradients and Hessians. The resulting gradient based algorithms often yield a good estimate for the sought solution in few iterations and can either be used directly or to quickly warm start existing algorithms.