Learning to traverse convective flows at moderate to high Rayleigh numbers

Abstract

We study the navigation of a self-propelled inertial particle in two-dimensional Rayleigh--B\'enard convection at Prandtl number $Pr = 0.71$ and cell aspect ratio $\Gamma = 4$ for Rayleigh numbers $Ra$ ranging from $10^{7}$ to $10^{11}$. A reinforcement-learning (RL) controller selects the propulsive acceleration, subject to an upper bound $\mathcal{A}_{\max}$, to achieve a prescribed horizontal displacement. We find that the success rate increases abruptly with $\mathcal{A}_{\max}$ at moderate $Ra$, whereas at higher $Ra$ the transition becomes more gradual and shifts to larger $\mathcal{A}_{\max}$. Moreover, although the completion time increases with $Ra$, the propulsion energy required for successful traversal decreases. Proper orthogonal decomposition (POD) reveals that these performance differences arise from reorganisation of the carrier flow. At moderate $Ra$, the dominant large-scale circulation partitions the domain through robust transport barriers, requiring a finite thrust surplus to cross them; at higher $Ra$, energy is distributed across many modes, the barriers fragment, and transient plume-assisted pathways emerge. Compared with a constant-heading baseline, the learned policy aligns with local currents and consumes significantly less energy. Lagrangian coherent structure (LCS) analysis further shows that the RL agent inherently learns to cross repelling barriers and surf along attracting pathways. Finally, by mapping these behaviours onto the local Eulerian flow topology using Voronoi tessellation and the $Q$-criterion, we distil an interpretable, physics-based heuristic strategy that achieves robust navigability. These results connect turbulent-flow organisation with autonomous navigation under bounded actuation.