Autonomous agents are increasingly thought of as team members alongside humans in large-scale operations. Both autonomous agents and humans have inherent advantages and limitations that might affect the outcome of an operation. The inclusion of autonomy within a team requires a significant effort to prepare autonomous agents to interact with human decision-makers. A fundamental challenge emerging is how to train humans and autonomous agents to achieve the operation goals with maximum performance. In this paper, we develop a game-based platform to train and quantitatively analyze the performance of human-autonomy teams in a disaster relief scenario. This framework consists of multiple interfaced components that allow a human decision-maker to collaborate with autonomous agents in an immersive virtual environment to accomplish a mission without cognitive overload. The disaster relief scenario is simulated in a high-fidelity environment using Unreal Engine where a human decision-maker interacts with the environment in real-time using a VIVE virtual reality headset and controllers. Autonomous agents possessing various capabilities are trained separately using a decentralized artificial intelligence algorithm to allocate tasks and collaborate with team members through reinforcement learning. Physiological data including pupil dilation, heart rate, and sweat gland activity are measured from the human during the time of the operation. This data is input to a trained machine learning architecture to monitor the cognitive task load and situation awareness of the human during the operation. To reduce the cognitive load of the human during operation, an adaptive user interface is constructed as a virtual tablet in the game engine to assist the human in decision-making depending on the cognitive load status. The developed game-based framework provides insights into the interactions of a human with multiple autonomous agents in time-critical operations. The immersive environment is a fundamental modeling and simulation tool for further developments and design of human-autonomy teams.
Keywords
AI,HUMAN FACTORS,SIMULATIONS
Additional Keywords
Virtual Reality, Adaptive User Interface, Human Autonomy Teaming, Task Allocation, Multi-agent Systems