Efficient Calculation of Regular Simplex Gradients

Abstract

Simplex gradients are an essential feature of many derivative free optimization algorithms, and can be employed, for example, as part of the process of defining a direction of search, or as part of a termination criterion. The calculation of a general simplex gradient in $\mathbb{R}^n$ can be computationally expensive, and often requires an overhead operation count of $\mathcal{O}(n^3)$ and in some algorithms a storage overhead of $\mathcal{O}(n^2)$. In this work we demonstrate that the linear algebra overhead and storage costs can be reduced, both to $\mathcal{O}(n)$, when the simplex employed is regular and appropriately aligned. We also demonstrate that a second order gradient approximation can be obtained cheaply from a combination of two, first order (appropriately aligned) regular simplex gradients. Moreover, we show that, for an arbitrarily aligned regular simplex, the gradient can be computed in only $\mathcal{O}(n^2)$ operations.