To RL or not to RL? An Algorithmic Cheat-Sheet for AI-Based Radio Resource Management

Abstract

Several Radio Resource Management (RRM) use cases can be framed as sequential decision planning problems, where an agent (the base station, typically) makes decisions that influence the network utility and state. While Reinforcement Learning (RL) in its general form can address this scenario, it is known to be sample inefficient. Following the principle of Occam's razor, we argue that the choice of the solution technique for RRM should be guided by questions such as, "Is it a short or long-term planning problem?", "Is the underlying model known or does it need to be learned?", "Can we solve the problem analytically?" or "Is an expert-designed policy available?". A wide range of techniques exists to address these questions, including static and stochastic optimization, bandits, model predictive control (MPC) and, indeed, RL. We review some of these techniques that have already been successfully applied to RRM, and we believe that others, such as MPC, may present exciting research opportunities for the future.