Evaluation of simulation-based training systems to determine their contributions to trainee proficiency and to determine the level of cost avoidance vice live training is essential to plan the future live-virtual-constructive training environment for the United States Marine Corps (USMC). This need is reinforced in a recent Government Accountability Office report (GAO 13-698, August 2013) on Army and Marine Corps Training titled Better Performance and Cost Data Needed to More Fully Assess Simulation-Based Efforts, which states that the Services “lack key performance and cost information that would enhance their ability to determine the optimal mix of training and prioritize related investments.� USMC Program Manager Training Systems (PM TRASYS) has conducted cost avoidance studies on USMC simulation-based training systems for the past 2 years, and these studies are being refined to capture improved cost information. A related study, begun in June 2013, evaluates the effects of USMC simulation-based training programs on proficiency.
This paper presents the process, results, and recommendations of the recent PM TRASYS Proof of Concept (POC) study of measuring proficiency changes and cost avoidance due to use of the M1A1 Advanced Gunnery Training System (AGTS) simulator. For the POC, a group of consistent crews in initial AGTS training are monitored (without interference) through a sequence of 10 gunnery table tasks, with a total of 500+ task instances in the AGTS simulator, to the culminating live-fire tasks. Early session scores are compared to “Gate-To-Live-Fire� scores in the simulator, and these results are compared to the live-fire M1A1 qualification scores for these crews. Results of the POC are promising. The study finds that with performance-oriented metrics and measures, tied to doctrine and captured automatically, it is possible to determine both proficiency trending and cost avoidance. This paper also discusses lessons learned and provides recommendations and implications of findings for training system design.