On the Duality between Feature and Sample Screening

Abstract

Feature and sample screening reduce the cost of machine learning by eliminating irrelevant features and noninformative samples, respectively. Although recognized as primal-dual counterparts, their relationship remains informal and model-dependent. Viewing screening and duality as transformations of objective functions, we introduce Fenchel-Rockafellar (FR) representations, a class of convex problems encompassing the Lasso and SVM that is closed under both transformations. We then prove that feature and sample screening form an equivariant pair: dualization followed by feature screening is equal to sample screening followed by dualization.