Reliability Testing of Applications on Windows NT

The ntDTS (Windows NT Dependability Test Suite) fault injection tool can be used to (1) obtain fault injection-based evaluations of system reliability; (2) compare the reliability of different applications, fault tolerance middleware, and platforms; (3) provide feedback to improve the reliability of the target applications, fault tolerance middleware, and platforms. This paper describes the architecture of the tool as well as the procedure for utilizing the tool. Data from experiments with the ntDTS tool used on the Apache web server, the Microsoft IIS web server, and the Microsoft SQL server, along with the Microsoft Cluster Server (MSCS) and Bell Labs watchd (part of NT-SwiFT) fault tolerance packages is presented to demonstrate the utility of the tool. The observations drawn from the data also illustrate the strengths and weaknesses of the tested applications and fault tolerance packages with respect to their reliability. The MSCS and watchd software are shown to be effective in increasing the reliability of the monitored applications, without incurring appreciable performance overheads. However, applications using watchdare better able to tolerate the set of injected faults. The data from the experiments was also used to make improvements to watchd. Using the ntDTS tool, reliability and performance comparisons between Apache and IIS show that Apache is better able to tolerate the set of injected faults than IIS, with or without additional fault tolerance middleware. Apache responds faster to client requests when faults are tolerated without application restart. However, the average response time for IIS is faster if an application restart is needed.