In 2014, companies had invested about $167 million into Autonomous Vehicle (AV) technologies. By 2019, these investments totaled more than $100 billion. The Pentagon’s 2020 fiscal year budget proposal included $3.7 billion for research and development of unmanned and autonomous technologies. Studies show that 52% of battlefield casualties occur when soldiers deliver food and other supplies in combat zones, and hence was theorized that the use of AVs could substantially mitigate such risks and save lives. However, AVs must be tested in a multitude of scenarios before they are practically viable for military and civilian applications. Physical AV data for testing are generally unavailable from commercial or military entities due to proprietary or security concerns. This makes simulations a feasible alternative to study them. However, creating AV simulations with the fidelity, scalability, and customization come with a number of research questions such as: how can AVs be trained for autonomous driving?, how can communication be established between different traffic management subsystems? and how can multiplayer collaboration be achieved?

A three-component visualization framework was developed to address the above challenges. First, multiple virtual vehicles were trained using machine learning techniques to autonomously drive within a specific road intersection scenario. Second, these virtual AVs were introduced to physical agents such as cars and bike riders. Third, the driving states of the physical agents and the AVs were synchronized using a client-server architecture with a traffic simulator that probabilistically generated vehicle and pedestrian traffic. The AVs and the physical agents appear as entities within the traffic simulator to which the generated traffic computes responses and are network synchronized to collectively form a multimodal traffic simulation system. Results from implementing and testing this framework in multiple scenarios show that properly trained AVs can serve as a proof-of-concept validation for developing military and civilian applications.