Software test automation provides hope for testers who face the daunting task of testing large, complex software systems. However, this hope often fades like a sunset as testing progresses from unit testing to system testing. One reason for this is that the interface between the system and its operating environment or the user becomes increasingly more complex as testing progresses from unit testing to system testing. In particular, the graphical user interface (GUI) has proven to be one of the most difficult challenges for test automation. This paper describes the efforts and difficulties of creating and using an automated test harness for the GUI of the U.S. Army's OneSAF Testbed Baseline (OTB) simulation system.

There is a significant amount of effort required for a user to create scenarios with the OTB user interface, even for simple scenarios. From a testing perspective, a test harness is needed to reduce this effort, thereby allowing more testing to be done within a given amount of time. Unfortunately, this generally requires that the test harness be custom made for the system being tested. This paper describes the methodology of creating a custom test harness for the OTB system. The goal in developing this test harness was to automate completely the process of creating scenarios in OTB using the GUI. This allows the tester to focus more on the execution of the scenario and determining its correctness. The test harness that was developed is capable of reading test scenarios provided by the tester, and then automates creating the scenario via the OTB user interface. Only partial test automation was achieved due to irregularities and complexities in the OTB user interface. This paper describes these problems and provides suggestions for improved user interface design strategies that can enhance possibilities for test automation of simulation system GUIs.