Abstract
Extended Reality (XR), an umbrella term for Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR), continues to be used in a wide variety of fields, including training, education, entertainment, and especially the military. The Army requested $427 million in 2024 for the Synthetic Training Environment program, which features XR for many of the desired applications. For complex and immersive XR environments, advanced features are often required, including hand, eye, and body tracking, custom physical interactions, and high visual fidelity. Industry, academic researchers, and the open-source community have started to introduce equipment and software tools that address these needs. However, achieving these capabilities in a seamless integrated simulation environment often requires hardware and software from different vendors or open-source platforms. Ensuring compatibility between heterogeneous equipment and software in an XR simulation presents significant challenges for developers.
This paper presents two case studies that demonstrate the challenges of integrating heterogeneous commercial and open-source equipment and software into a single XR simulation. The development process used open-source software including OpenXR and a local laptop in a closed WiFi network to overcome the hardware and software issues. The first case is a portable MR welder training simulator that supports interaction with a customized welder frame interface and gun. This application requires integrating an Arduino, multiple Vive Ultimate Trackers, and standalone XR headsets. The second case is a training application for veterinary euthanasia using captive bolt guns. This application requires the use of HMDs with Vive Ultimate Trackers to enable interaction with a physical bolt gun in an XR simulation across multiple headsets. A technical evaluation of both systems that was conducted, will be discussed, measuring several performance metrics including frame rate and latency. These use cases highlight the potential for developing complex XR environments using heterogeneous commercial and open-source equipment and software.
