RL+AHP: A Novel Reinforcement Learning driven AHP for Slice Aware mode selection in D2D enabled Heterogeneous Networks

Abstract

The mode selection problem in device-to-device communication (D2D) enabled Fifth generation (5G) heterogeneous networks (HetNet) aims prioritizing four key performance indicators (KPIs) namely data rate, latency, reliability and jitter across three slices: enhanced mobile broadband (eMBB), ultra reliable low latency (uRLLc) and massive machine type communications (mMTC). Such priority assignment must be \emph{traded off} among three access technologies, i.e., Long Term Evolution advanced (LTE-A), New Radio (NR) and D2D, while minimizing handover frequency. In existing mode selection approaches for HetNet, slice specific quality of service (QoS) requirements are largely ignored. In this work, a novel mode selection algorithm is proposed by combining a two level Analytic Hierarchy Process (AHP) with a Reinforcement Learning (RL) method. While the two level AHP facilitates decision making based on multiple criteria (i.e., KPIs) and options (i.e., LTE-A, NR, D2D mode), the RL approach computes the weights of each criteria based on the feedback from the environment. Simulation results show that our proposed algorithm outperforms related works in terms of the major KPIs for all three slices. For eMBB applications, our approach increases throughput by $33\%$; for uRLLc applications, our approach significantly decreases latency and BER ($27\%$ and $10\%$ respectively) and for mMTc applications, our approach significantly decreases latency ($44\%$). Moreover, it has been shown that the proposed RL+AHP approach outperforms the existing DRL based approaches in terms of CPU usage when the number of criteria is reasonably low ($<6$).