Technical Report: Partially Asynchronous Distributed Optimization on Riemannian Manifolds

Abstract

There is a large body of work on distributed optimization for problems in Euclidean space. However, many existing algorithms and analyses do not directly transfer to problems defined on smooth manifolds. Therefore, in this paper we propose a distributed asynchronous optimization algorithm for problems on Riemannian manifolds. The algorithm we develop is a parallelized form of Riemannian block coordinate descent in which all agents can compute and communicate asynchronously. We assume only "partial asynchrony", namely that all delays in computations and communications obey some finite bound. We relax common assumptions by requiring neither convexity of the objective nor compactness of the manifold. Under mild conditions, we prove that agents' iterates drive the Riemannian gradient of their objective to zero, and we derive a rate of convergence for the norm of the gradient. Under different communication delays and numbers of agents over a finite horizon, we empirically show that our algorithm achieves a suboptimality of up to 730 times smaller than that of a synchronous baseline algorithm in an example simulation.