Advances in artificial intelligence (AI) such as natural language processing (NLP) and reinforcement learning (RL) are enabling a resurgence in immersive flight training for both instructor-led and self-paced training, such as with immersive MR/VR (Mixed Reality/Virtual Reality) devices.

Introduction of MR/VR will likely require dialog management and command & control capability in a self-paced learning context. Adding an NLP-based cognitive agent acting as a virtual instructor and co-pilot provides the required immersion level to broaden the spectrum of self-paced learning during the pilot’s learning journey. Pilots receive instant feedback on their performance, explanations of the communication procedures, and progress tracking as they develop their skills.

Accent-tolerant advances in speech interaction are used to recognize radio transmissions from the ownship using NLP on a flight training knowledge base. Conversation agents increase student immersion and offer more realistic workloads by fully automating the ATC (air traffic control) function, freeing up the instructor to focus more on core observation and training tasks, and allowing more automated flying without an instructor for MR/VR simulation training.

The complexity of real-world ATC communications, such as conditional clearances and instructions that include give-way, can only be simulated when the ownship is fully embedded with other traffic. An AI ATC module leverages a collaborative multi-agent framework to manage air traffic during real-time MR/VR flight simulation in a synthetic urban environment scenario.

This paper will explore issues around robustness in reinforcement learning and evolutionary optimization problems, alongside new results in collaborative multi-agent systems. These outputs will provide further results in nonlinear function approximation (e.g. deep neural networks) and optimization methods in stochastic environments. It will also study human factors during immersive Mixed-Reality training of emergency scenarios with an AI agent integrated into an eVTOL (electric Vertical Takeoff and Landing aircraft) flight training simulation and operation platform.