Main Subcommittee Category: Simulation

How would you label your submission?: Academia

Abstract:

The Royal Netherlands Air Force (RNLAF) is revolutionizing its tactical training capabilities by creating a state-of-the-art Multi-Ship-Multi-Type (MSMT) simulator center at Gilze-Rijen Air Force Base. This initiative necessitates the development of high fidelity models such as a virtual Auxiliary Power Unit (APU) for the CH-47F Chinook helicopter's Rear Crew Trainer (RCT) simulator. However, traditional methods of 3D modeling are resource-intensive and time-consuming, posing a challenge for high fidelity virtual model development. In this paper we propose using Artificial Intelligence (AI) techniques, including Surface-Aligned Gaussian Splatting (SuGaR) and object segmentation, to efficiently generate 3D meshes from a set of pictures of real-world objects, addressing the main challenge faced by the RNLAF.

We outline criteria for meshes to be compatible with simulators, focusing on polygon count and fidelity, and evaluate the capability of various experimental AI techniques in meeting these requirements. As the resulting objects have to be isolated from the environment, we examine AI segmentation techniques to automate this. We present our findings by developing the virtual APU for the RCT, which will demonstrate that these methods offer a favorable trade-off between quality, reduced labor, and costs.

Using experimental AI is challenging due to the demand for highly specialized skills and the plethora of techniques involved. To increase accessibility, we build on our lessons learned and propose an interactive 'mesh-as-a-service' platform to facilitate the integration of real-world objects into simulated environments.

Our research demonstrates that AI-driven methods are transforming the landscape of 3D mesh generation, presenting a scalable and efficient alternative to traditional approaches. The development of ‘mesh-as-a-service’ will make the creation of digital twins and the rapid construction of immersive virtual training scenarios more accessible. These innovations have the potential to set new standards in military simulation and training, with implications that extend beyond the RNLAF to the defense sector worldwide.

Keywords: 3D SCANNING;AI;AUTOMATION;IMMERSIVE;M&S AS A SERVICE;MACHINE LEARNING;PROTOTYPING;RAPID MODELING;SCALABILITY;SIMULATORS;VIRTUAL