Abstract
The Department of Defense (DOD) increased funding for Artificial Intelligence (AI) from $874 million in 2022 to approximately $1.8 billion in 2024. The allocation demand signifies a strategic push for machine learning (ML) driven capabilities, which includes Autonomous Vehicle (AV) control and navigation in a variety of military applications. However, physical tests of AVs incur high development costs and introduces substantial safety risks. Extended Reality (XR) simulation platforms provide a safe and cost-effective alternative to determine viability and repeat testing but may significantly differ from real-world driving. If XR could be proven as a viable platform for traffic data collection, driving scenarios, such as AVs, could be studied extensively. A custom multimodal traffic simulation platform (i.e., InterchangeSE) was utilized to simulate various driving scenarios. InterchangeSE possesses four key capabilities: autonomous vehicle integration, 3D virtual environment rendering, human agent interfacing, and traffic generation. InterchangeSE was used to collect driving data to train and deploy ML models for AV control (i.e., steering angle and throttle response).
This paper presents the training, deployment, and evaluation of various ML models in the XR simulation, InterchangeSE. Two scenarios, a baseline road environment with and without traffic, were used for data collection. Using Design of Experiments (DOEs), five ML end-to-end learning architectures (i.e., Convolutional Neural Networks and lightweight transformers) were subjected to variability in their network design and common image augmentation techniques, such as horizontal flipping, brightness variability, additional noise, and shadow manipulation. Alongside investigating model training, a detailed statistical analysis was performed to assess the impacts of these model configurations under the real-time constraints imposed by the virtual environment. The top performing models were deployed and evaluated in the XR simulation platform under different driving scenarios, including three-lane highways, highway merging, and intersections, with the findings yielding effective AV control.