A Refined Proximal Algorithm for Nonconvex Multiobjective Optimization in Hilbert Spaces

Abstract

This paper is devoted to general nonconvex problems of multiobjective optimization in Hilbert spaces. Based on Mordukhovich's limiting subgradients, we define a new notion of Pareto critical points for such problems, establish necessary optimality conditions for them, and then employ these conditions to develop a refined version of the vectorial proximal point algorithm with providing its detailed convergence analysis. The obtained results largely extend those initiated by Bonnel, Iusem and Svaiter \cite{Bonnel2005} for convex vector optimization problems and by Bento et al. \cite{Bento2018} for nonconvex finite-dimensional problems in terms of Clarke's generalized gradients.