Although the task of performing an empirical Training Effectiveness Evaluation (TEE) is not trivial in any domain, the nature of military training makes it a particularly difficult arena. Specifically, in operational environments, the lack of experimental control, limited participants, and extreme expense and resource requirements all lead to challenges for evaluators investigating the degree that the training system addresses targeted goals.

A requirements-driven approach to TEE aimed at facilitating a less resource intensive evaluation has been designed. Advantages of this approach include the capability to be conducted early in the design cycle, allowing results to be fed back into the iterative design of the system. This approach leverages Sensory Task Analysis (STA) outputs to identify the multisensory information and interaction capabilities necessary to support the task, which are then used to evaluate the degree that the system supports these requirements. From this, redesign recommendations are developed, identifying specific system upgrades that would allow a greater percentage of training objectives to be targeted. This approach has been used successfully in the evaluation of multiple operational United States Marine Corps training systems including a desktop Fire Support Team (FiST) team training system, a high fidelity Forward Area Controller (FAC) training system, and a mixed reality infantry patrol trainer. This paper will discuss the method of conducting requirements-driven TEEs and initial validation results.

Susan Dean

Affiliation

Refine Results

Challenges of Scenario Design in a Mixed-Reality Environment

Cue Fidelity Evaluation: A Requirements-driven Approach to Training Effectiveness Evaluation