Env2Vec: Testing Virtual Network Functions at Scale via a Single Deep Learning Model

Performance testing of virtual network functions (VNFs) is problematic for software testing engineers, given the wide variety of deployment environments for the VNFs. By environment, we refer to a combination of aspects that influence performance such as hardware, middleware, software version, build, enabled features, network topol- ogy, and test case, to name a few. One common approach to automate the testing process uses machine learning (ML) and consists of two steps: (1) learning a resource characterization model for each VNF as a function of traf- fic using historical data where no performance problem occurred, and; (2) using the model to predict the resource utilization with respect to current traffic metrics during a new test. A performance problem is reported if the pre- dicted value deviates significantly from the observed re- source utilization. Previous ML approaches propose train- ing and updating a model for each combination of appli- cation and environment, which is not feasible, due to the problem of combinatorial explosion. To address this problem, we propose a single charac- terization model called Env2Vec, short for environment to vector, that uses deep learning to map environment pa- rameters to vector embeddings. These vector embeddings not only provide insight into the similarites of the various environments, but allow the model to extrapolate predic- tions to unseen environments: an action not possible using previous approaches. A comprehensive evaluation over large, real-world data sets, shows that our proposed ap- proach can achieve state-of-the-art performance for char- acterizing CPU utilization of VNFs, as well as a false alarm rate about 23% lower than baseline approaches for the downstream task of detecting performance problems, despite using a single model for all environments.