Monitoring Time-Varying Network Streams Using State-Space Models

Many network measurements (such as traffic counts or network delays) exhibit daily or weekly cyclical patterns. At the same time, these cyclical patterns change over time as circumstances change from day to day. Finding anomalous events in these data is non-trivial due to the time-varying nature of the expected normal behavior. In this paper, we develop an {em online} monitoring methodology for the time-varying cyclical streams of network data, that combines a baseline {em state-space model} and {em statistical control schemes} to monitor departures from the baseline model. The state-space model characterizes the normal evolution of the time series data: an observation equation captures the daily/weekly patterns using splines, and a state equation captures the normal changes in the daily/weekly patterns. Parameters of the state space models are initialized based on the training data, and updated for each incoming observation. The statistical control schemes for monitoring are designed based on forecasting errors from the baseline model, under the framework of statistical change detection. We demonstrate the effectiveness of our methodology using measurement data from an EVDO wireless network.