Given the increasing complexity of team simulations, there is a need to aid instructors by providing tools that can facilitate After Action Review (AAR). Without these tools, instructor evaluation and feedback might not be optimized due to human limitations in attention and memory (e.g., capability to simultaneously monitor and remember intricacies of all team members' performance). By providing a tool which can facilitate detection of root-cause errors, the diagnostic value of the AAR or system feedback provided to trainees may be improved, potentially increasing training efficiency by targeting the specific skills that need improvement.

The Performance ASsessment and diagnostic Tool (PAST) addresses these needs by 1) presenting instructors with the pertinent type and amount of information to properly assess training performance and 2) providing trainees in a deployed setting, with a "Trainer in a Box", allowing them targeted feedback about specific errors on which they need to focus. PAST achieves these goals through 1) performance measurement via system collected and semiautomated observer collected metrics, 2) performance diagnosis of root cause via metric relationship logic, 3) interpretation of performance trends through metric linkages to relevant training goals/objectives, and 4) intuitive and usable presentation of performance diagnosis and interpretation to facilitate an understanding of the root-cause of performance decrements and areas in need of improvement. PAST was developed in close collaboration with United States Marine Corps (USMC) subject matter experts (SMEs) and validated with instructors at a range of different schoolhouses, including a series of evaluations testing its functionality and usability. This paper discusses PAST functionality, evaluation and testing data collection with SMEs, results and the redesign resulting recommendations.