There are a variety of approaches to validation of simulation systems, each attempting to solve one or more specific issues. When viewed as a whole, these approaches can address a broad spectrum of issues. However, regardless of the validation approach, scalability and input domain explosion make it impossible to exhaustively test simulation systems. Improved methods are needed.

This paper describes a case study that demonstrates the application of statistical usage testing methods to provide quantitative support for test planning and user testing of a commercially available combat simulation game. First, typical approaches to validation of simulation systems are discussed with respect to some of the challenges of simulation system validation. Then the case study is used to provide a brief introduction to statistical usage testing and to illustrate the applicability of usage modeling and statistical usage testing to improve the efficiency and effectiveness of test planning, management, and analysis of test results. Results from the case study include suggestions for applying usage model analysis methods to iteratively improve usage models and support more efficient and effective achievement of specific test objectives.