Optimal Navigation in Stochastic and Disordered Gridworlds

Abstract

Navigation in complex and noisy environments is a key issue in diverse fields from biology to engineering. Despite extensive progress in numerical optimization methods for computing navigation policies, insights into how disorder reshapes optimal navigation remain elusive. To address this question, we investigate the navigation of a Brownian particle in a disordered energy landscape, modeled as a lattice with randomly distributed traps. Using dynamic programming, we compute the optimal navigation policies that minimize the mean first-passage time to a target site. To quantify the impact of disorder, we introduce a density of change from a Kullback-Leibler divergence, which captures how the optimal policy is reshaped by either the presence of disorder or the knowledge of its configuration. Our results reveal a non-monotonic dependence of the change of the policy on trap concentration, with a pronounced maximum. In the fluctuation-dominated regime where the navigation bias is weak, we derive an analytical expression for the density of change, and demonstrate that the maximum occurs unexpectedly at low trap concentrations.