A Newton-Like Algorithm for Likelihood Maximization: The Robust-Variance Scoring Algorithm

Abstract

This article studies a Newton-like method already used by several authors but which has not been thouroughly studied yet. We call it the robust-variance scoring (RVS) algorithm because the main version of the algorithm that we consider replaces minus the Hessian of the loglikelihood used in the Newton-Raphson algorithm by a matrix $G$ which is an estimate of the variance of the score under the true probability, which uses only the individual scores. Thus an iteration of this algorithm requires much less computations than an iteration of the Newton-Raphson algorithm. Moreover this estimate of the variance of the score estimates the information matrix at maximum. We have also studied a convergence criterion which has the nice interpretation of estimating the ratio of the approximation error over the statistical error; thus it can be used for stopping the iterative process whatever the problem. A simulation study confirms that the RVS algorithm is faster than the Marquardt algorithm (a robust version of the Newton-Raphson algorithm); this happens because the number of iterations needed by the RVS algorithm is barely larger than that needed by the Marquardt algorithm while the computation time for each iteration is much shorter. Also the coverage rates using the matrix $G$ are satisfactory.