Background

The adoption of mixed-reality (MR) simulators by the Naval Aviation Enterprise (NAE) necessitates a comprehensive understanding of the impact of MR technology on human factors. Unlike traditional projected-screen simulators, MR simulators utilize head-mounted displays (HMDs) to present virtual environment visuals in close proximity to the pilot's eyes. This departure from conventional methods raises concerns regarding the distortion of natural human depth perception and distance estimation, thereby highlighting the critical human factor of stereoscopy on performance efficiency and cognitive load.

Two primary types of MR HMDs exist: optical see-through (OST) and video see-through (VST). In OST, users perceive actual objects in the physical environment, with virtual objects overlaid on a semi-transparent display within their field of view. Conversely, VST captures video from the user's viewpoint, integrating virtual objects with this video and presenting the composite imagery on screens in close proximity to the user's eyes. Notably, in OST, users must adjust their focal distance when transitioning between viewing virtual and real objects, whereas in VST, all objects are rendered on screens, eliminating the need for focal adjustments. However, no prior research has explored whether this distinction influences cognitive workload or susceptibility to cybersickness.

Significance

This study the enhances of NAE's understanding of MR technology, facilitating the development of more effective MR simulators, delineating their limitations, and offering insights into the integration of MR simulators into flight training curricula.

Methods

We assessed performance disparities among approximately thirty subjects using a version of the NASA Multiple Attribute Test Battery. Each subject engaged in tasks under three randomly assigned conditions:

OST MR utilizing a Hololens
VST MR employing a Varjo XR-4 FE.
Traditional legacy systems.

Subjects also completed a simulation sickness questionnaire to measure cybersickness effects.

Results

The experiment is presently ongoing, with data collection and analysis slated for completion by May 2024.

Jon Vogl

Affiliation

Refine Results

Assessing Cognitive Workload in Mixed Reality Flight Simulators for Naval Aviation