A Mini-Batch Counterexample to Last-Iterate Convergence in Definable Optimization

Abstract

We give a counterexample to the convergence conjecture in Remark 12 of [Bolte & Pauwels, 2021] for mini-batch stochastic approximation with definable potentials. The construction uses two convex piecewise-affine, hence semialgebraic, summands on $\mathbb{R}$. We choose a deterministic nonincreasing block stepsize sequence satisfying $α_k = o(1/\log k)$ and an admissible minimum-norm selection from each aggregate batch field. On successive blocks, the iterates form lazy reflected random walks on nested dyadic lattices. An explicit endpoint-cover-time estimate, Markov's inequality, and the first Borel-Cantelli lemma imply that almost surely every sufficiently late block's iterates visit their entire lattice. Consequently, the iterates remain in $[-1,1]$ but do not converge, and their accumulation set is exactly $[-1,1]$, on which the averaged objective is constant. Finally, the construction has $\sum_k α_k^2 =\infty$. Both Chat-GPT 5.6 (Sol) and Gemini Pro 3.1 (DeepThink) were used in the development and drafting of this result.