Under-determined linear systems and $\ell_q$-optimization thresholds

Abstract

Recent studies of under-determined linear systems of equations with sparse solutions showed a great practical and theoretical efficiency of a particular technique called $\ell_1$-optimization. Seminal works \cite{CRT,DOnoho06CS} rigorously confirmed it for the first time. Namely, \cite{CRT,DOnoho06CS} showed, in a statistical context, that $\ell_1$ technique can recover sparse solutions of under-determined systems even when the sparsity is linearly proportional to the dimension of the system. A followup \cite{DonohoPol} then precisely characterized such a linearity through a geometric approach and a series of work\cite{StojnicCSetam09,StojnicUpper10,StojnicEquiv10} reaffirmed statements of \cite{DonohoPol} through a purely probabilistic approach. A theoretically interesting alternative to $\ell_1$ is a more general version called $\ell_q$ (with an essentially arbitrary $q$). While $\ell_1$ is typically considered as a first available convex relaxation of sparsity norm $\ell_0$, $\ell_q,0\leq q\leq 1$, albeit non-convex, should technically be a tighter relaxation of $\ell_0$. Even though developing polynomial (or close to be polynomial) algorithms for non-convex problems is still in its initial phases one may wonder what would be the limits of an $\ell_q,0\leq q\leq 1$, relaxation even if at some point one can develop algorithms that could handle its non-convexity. A collection of answers to this and a few realted questions is precisely what we present in this paper.