Exploring Reinforcement Learning for Scheduling in Cellular Networks

Cellular network scheduling is crucial for wireless deployments like 4G, 5G, and 6G and is a challenging resource allocation task performed by the scheduler located at the base stations. The scheduler must balance two critical metrics, throughput and fairness, which often conflict, as maximizing throughput favors users with better channel conditions, while ensuring fairness requires allocating resources to those with poorer channel conditions. The proportional fairness metric is a prominent scheduling approach that aims to balance these competing metrics with minimal compromise. The common strategy to attain proportional fairness relies on a greedy approach in which each resource block is allocated to the user who maximizes the proportional fairness criterion. With such a strategy, the scheduler can ensure that the resources allocated to the users at each time instance maximize the proportional fairness metric. However, users can usually tolerate some delay and are willing to accept temporary fairness imbalances if they ultimately improve their performance, provided that the fairness criterion is maintained over time. In this paper, we propose a new scheduler that uses reinforcement learning to enhance proportional fairness. The suggested scheduler considers both current and predicted future channel conditions for each user, aiming to maximize the proportional fairness criterion over a set of predefined periodic time epochs. Specifically, by learning patterns in channel fluctuations, our reinforcement learning-based scheduler allocates each resource block not to the user who maximizes the instantaneous proportional fairness metric, but to the user who maximizes the expected proportional fairness metric at the end of the current time epoch. This approach achieves an improved balance between throughput and fairness across multiple slots. Simulations demonstrate that our approach outperforms standard proportional fairness scheduling. We further implemented the proposed scheme on a live 4G eNodeB station and observed similar gains.