Engagement is a principal factor in learning retention, but is difficult to directly observe and measure in many training settings, including virtual reality (VR) simulation. Training using VR is gaining broad adoption across DoD. In the Air Force, for instance, a new approach to Specialized Undergraduate Pilot Training (SUPT) called Pilot Training Next (PTN) integrates traditional flying sorties with VR-enabled ground-based training devices to accelerate training and improve readiness. To support PTN with metrics of attention and engagement, Eduworks and USC’s Institute for Creative Technologies partnered with the Air Force Research Laboratory (AFRL) to develop machine learning (ML) models that can measure user engagement during any computer-mediated training (simulation, courseware) and offer recommendations for restoring lapses in engagement. We developed and tested this approach, called the Observational Motivation and Engagement Generalized Appliance (OMEGA) in a PTN context. Two factors motivate this work. First, a goal of PTN is for an instructor pilot (IP) to simultaneously monitor multiple simulator rides. Being alerted to distraction, attention and engagement can help an IP manage multiple students at the same time, with recommendations for restoring engagement providing further instructional support. Second, the virtual environment provides a rich source of raw data that ML models can use to associate user activity with user engagement. We created a testbed for data capture in order to construct the ML models, based on theoretical foundations we developed previously. We ran pilots through multiple PTN scenarios and collected formative data from instructors to evaluate the utility of the recommendations OMEGA generates regarding how lapsed engagement can be restored. We present findings that validate the use of ML models for detecting engagement from the data characteristic of virtual environments. These findings, though preliminary, will support innovating and accelerating conventional and VR applications as training adapts to an unexpected future.
Training
Attention and Engagement in Virtual Environments: Measuring the Unobservable
21 views