This paper combines cognitive task analysis and expert input to design and develop a framework for assessing learner competencies and performance across multiple training scenarios. We adopt a hierarchical Bayesian approach to aggregate information from multiple modalities to derive competency metrics that relate to team coordination and individual psychomotor, cognitive, and affective measures of performance. The unified framework is represented as a task model that maps onto multiple task domains. The resulting hierarchical competency structure connects observed low-level performance measures for each task domain into higher-level competencies that are common across domains. By utilizing Bayesian inference to propagate evidence up the competency model, our framework is able to build a common model of high-level learner cognitive and psychomotor performance using evidence from multiple independent tasks. We demonstrate the effectiveness of the proposed framework using a case study of groups of soldiers performing two dismounted battle drills, and show that the performance displayed by the soldiers provides consistent evidence for their higher-level competency states. With continued research and development, the proposed framework could allow for consistent longitudinal assessment of trainees based on observable evidence across a wide variety of domain skills and tasks.

Keywords
BAYESIAN NETWORKS, COGNITIVE, COMPETENCY BASED TRAINING, FUSION , HUMAN PERFORMANCE, LEARNING ANALYTICS, MACHINE LEARNING, TEAM TRAINING

Additional Keywords