On stepwise regression

Abstract

Given data $y$ and $k$ covariates $x$ one problem in linear regression is to decide which in any of the covariates to include when regressing $y$ on the $x$. If $k$ is small it is possible to evaluate each subset of the $x$. If however $k$ is large then some other procedure must be use. Stepwise regression and the lasso are two such procedures but they both assume a linear model with error term. A different approach is taken here which does not assume a model. A covariate is included if it is better than random noise. This defines a procedure which is simple both conceptually and algorithmically