Reinforcement Learning with thermal fluctuations at the nano-scale

Abstract

Reinforcement Learning offers a framework to learn to choose actions in order to achieve some goal. However, at the nano-scale, thermal fluctuations hamper the learning process. We analyze this regime using the general framework of Markov Decision Processes, which applies to a wide variety of problems from nano-navigation to nano-machine actuation. We show that at the nan-oscale, while optimal actions should bring an improvement proportional to the small ratio of the applied force times a length-scale over the temperature, the learned improvement is smaller and proportional to the square of this small ratio. Consequently, the efficiency of learning, which compares the learning improvement to the theoretical optimal improvement, drops to zero. Nevertheless, these limitations can be circumvented by using actions learned at a lower temperature. These results are illustrated with simulations of the control of the shape of small particle clusters.