Joint Optimization and Statistical Inference for Zero-th Order Simulation Optimization

Abstract

We consider stochastic optimization problems with the dual tasks of (i) effectively finding the optimizer and (ii) reliably conducting statistical inference for the optimal objective function value. We find that classical simulation optimization and stochastic optimization algorithms, despite of their fast convergence rates to the optimizer under strong convexity assumptions, may not come with a valid central limit theorem (CLT) with a vanishing bias. This non-vanishing bias can harm statistical inference and the construction of asymptotically valid confidence intervals. We fix this issue by providing a new stochastic optimization algorithm that on one hand maintains the same fast convergence rate and on the other hand permits the establishment of a valid CLT with vanishing bias. We discuss practical implementations of the proposed algorithm and conduct numerical experiments to illustrate the theoretical findings.