OnSlicing: Online End-to-End Network Slicing with Reinforcement Learning

Network slicing allows mobile network operators to virtualize infrastructures and provide customized slices with isolated resources for supporting various use cases with heterogeneous requirements. Reinforcement learning (RL), especially online RL, has showed promising potential in optimizing real system directly and eliminating the simulation-to-reality discrepancy. Optimizing cross-domain resources with online RL is however challenging, as random exploration of RL violates service level agreements (SLAs) of slices and fixed central RL design cannot adapt to dynamic networks. In this paper, we propose OnSlicing, an online end-to-end network slicing system, to optimize resource utilization for supporting various slices. We design a novel online distributed resource orchestration algorithm, in which slice SLAs are maintained during online learning by using constraint-aware policy update method and baseline switching mechanism, and dynamic network adaptation is ensured by the unique design of action projection and parameter coordination. We design an end-to-end network virtualization platform with multiple new domain managers in both RAN, TN, CN and Edge, which allows dynamic resource reconfiguration in seconds. We implement OnSlicing on an end-to-end slicing testbed designed by using OpenAirInterface in both 4G LTE and 5G NR, SDN managed by OpenDayLight, and edge computing platform. The experimental results show that OnSlicing reduces 34.01% average resource utilization with nearly zero violations (0.02%) of slice SLA as compared to prior solutions all through the online learning phase, and achieves 30% utilization reduction without any SLA violation after policy learned.