An overview of SPSA: recent development and applications

Abstract

There is an increasing need in solving high-dimensional optimization problems under non-deterministic environment. The simultaneous perturbation stochastic approximation (SPSA) algorithm has recently attracted considerable attention for solving high-dimensional optimization problems where the analytical formula cannot be attained. SPSA is designed to estimate the gradient by applying perturbation on a random subset of dimensions at each iteration. SPSA can be easily implemented and is highly efficient in that that it relies on measurements of the objective function, not on measurements of the gradient of the objective function. Since its invention, SPSA has been implemented in various fields such as reinforcement learning, production optimization etc. The paper briefly discuss the recent development of SPSA and its applications.