Automatic Ex-Vivo Regression Testing of Microservices

The microservice-based architecture can be conveniently used to implement applications composed of multiple services that can evolve independently. Changed services are normally tested against regression test suites, which aim to reveal faults before services are deployed in the field. These test suites are however designed in-house by developers who cannot predict how services will be used in the field. The available regression test suites may thus easily miss relevant test scenarios, failing to prevent the deployment of faulty services. To address this problem, this paper proposes ExVivoMicroTest, an approach that exploits data extracted from the field to generate non-regression test cases that can be used to validate changes while taking actual usage scenarios into consideration. To achieve isolation and the faithful reproduction of interactions observed in the field, ExVivoMicroTest exploits cloud technologies, containers in particular, to generate a mocked environment that fully isolates a microservice under test from the rest of the system. The mocked microservices present in this environment are instructed to exactly replay the interactions observed in the field, enabling the reproduction of field executions. Testing microservices based on information extracted from the field can improve the effectiveness of the validation activity making the service update process faster and more reliable. Our evaluation with the microservices in PiggyMetrics shows that ExVivoMicroTest can be feasibly used to reveal faults based on automatically collected data.