Augmenting Practical Cross-layer MAC Schedulers via Offline Reinforcement Learning

An automated offline design process for optimized cross-layer schedulers can produce augmented scheduling algorithms tailored to a target deployment scenario. We discuss the application of ODDS, a reinforcement learning technique we introduced, for augmenting LTE MAC scheduler algorithms of practical significance. ODDS observes the correlation between the value of a utility function and the parameters applied by an instrumented baseline scheduler, selecting the best variable ranges and sets of parameters via an offline Monte Carlo exploration of the problem space. The result of the ODDS process is a compact definition of a scheduling policy that has been optimized for a target scenario and utility function. In this paper we instrument a production scheduler definition to evaluate the potential of augmented schedulers in practical applications, and experiment with awareness to traffic classes by using a multi-class utility function, yielding scheduling policies that behave differently depending on the properties of individual traffic flows.