The Gradient descent method from the perspective of fractional calculus

Abstract

Motivated by gradient methods in optimization theory, we give methods based on $\psi$-fractional derivatives of order $\alpha$ in order to solve unconstrained optimization problems. The convergence of these methods is analyzed in detail. This paper also presents an Adams-Bashforth-Moulton (ABM) method for the estimation of solutions to equations involving $\psi$-fractional derivatives. Numerical examples using the ABM method show that the fractional order $\alpha$ and weight $\psi$ are tunable parameters, which can be helpful for improving the performance of gradient descent methods.