Truncated Stochastic Approximation with Moving Bounds: Convergence

Abstract

In this paper we propose a wide class of truncated stochastic approximation procedures with moving random bounds. While we believe that the proposed class of procedures will find its way to a wider range of applications, the main motivation is to accommodate applications to parametric statistical estimation theory. Our class of stochastic approximation procedures has three main characteristics: truncations with random moving bounds, a matrix valued random step-size sequence, and dynamically changing random regression function. We establish convergence and consider several examples to illustrate the results.