Unsupervised Anomaly Detection and Root Cause Analysis in Mobile Networks

Modern telecommunication networks are designed for high reliability; however, when they do fail or encounter performance degradation, it is difficult to detect and diagnose problems in a timely manner. Often the problems are subtle, and not catastrophic, which can be difficult to detect with typical KPIs and threshold based methods. Telecommunication networks are characteristically complex and, in the case of mobile networks, generate large amounts of data. Within the overall objective of network anomaly detection, the focus of this paper is on three aspects: dimensionality reduction through feature selection to determine the minimum set of features that provide the most information regarding the network state; the application of the multivariate unsupervised learning technique, Principal Component Analysis (PCA), to detect anomalies with low detection latency; and root cause analyses using finite state machines. Given that the number of anomalies is small in mobile networks, we employ an unsupervised approach for anomaly detection. For anomaly detection, we apply PCA on the normal data, and find the subspace where the variation of the normal data is small. By characterizing the variation of the normal data in the subspace, we derive a boundary of the normal data, and develop an anomaly detection model based on it. Once anomalies are detected, we analyze their message patterns for root cause analysis. Specifically, we compare the message patterns of the anomaly data to those of the normal data to determine where the problems are occurring. Moreover, we examine the error codes in the anomaly data to better understand the underlying problems. The algorithms are developed and tested with Per Call Measurement Data (PCMD) records from a 4G-LTE network. The impact of our work is the proactive detection and root cause analysis of network anomalies in mobile networks, thereby improving network reliability and availability.