Abstract
Modern military forces increasingly introduce simulation-based training to develop and sustain critical skills through procedure, tactical, or mission training in realistic threat environments. However, effectively assessing trainee performance remains a challenge. Most simulation systems include data recording, monitoring, or debrief/AAR tools to support trainee performance measurement and assessment. While these tools capture relevant data required for performance measurement, they fail to translate it into objective actionable insights for human assessors such as instructors. The usage of captured data is often limited to basic visualizations or communication replays, leaving contextual interpretation and analysis with respect to trainee task performance to the human assessor.
In this paper, we present a lightweight measurement framework for real-time context-based performance analytics to support trainee assessment in simulation-based training. The framework supports three key features. First, it is based on the chaining of individual measures to create a network of progressively rich information, ranging from (1) ground-truth data to (2) task-specific performance metrics and (3) objective assessments. Second, it supports the contextual activation of measures such that measurements can be delivered in real-time at the right time, relevant to the current task, situation, or mission phase. Finally, we show how AI techniques, such as Deep Learning for behavioral recognition, NLP for communication analysis, and rule-based methods for procedural evaluation, can enhance performance assessment. The outcome of measures can be visualized and delivered to human assessors by means of a dashboard, or fed into instructional systems (e.g. as xAPI statements) for learner profile management.
We evaluate the framework in a case study for fighter pilot training. Based on a use case scenario, we demonstrate how our framework enhances instructor capabilities by providing real-time, data-driven insights into pilot performance across key skill areas, including communication, tactical decision-making, and maneuvering.