Remaining competitive in future conflicts with technologically-advanced competitors requires us to accelerate our research and development in artificial intelligence (AI) for wargaming. More importantly, leveraging machine learning for intelligent combat behavior development will be key to one day achieving superhuman performance in this domain—elevating the quality and accelerating the speed of our decisions in future wars. Although deep reinforcement learning (RL) continues to show promising results in intelligent agent behavior development in games, it has yet to perform at or above the human level in the long-horizon, complex tasks typically found in combat modeling and simulation. Capitalizing on the proven potential of RL and recent successes of hierarchical reinforcement learning (HRL), our research is investigating and extending the use of HRL to create intelligent agents capable of performing effectively in these large and complex simulation environments. Our ultimate goal is to develop an agent capable of superhuman performance that could then serve as an AI advisor to military planners and decision makers. This research is primarily divided into five research areas aimed at managing the exponential growth of computations that have thus far limited the use of AI in combat simulations: (1) developing an HRL agent architecture for combat units; (2) developing a scalable HRL training framework for agents; (3) developing space-invariant state and action abstractions to manage the exponential growth of computations; (4) developing hybrid AI behavior models that leverage RL, expert systems, and game theory techniques; and (5) implementing our HRL agent architecture and training framework into both a low-fidelity and a high-fidelity combat simulation. This research will further the ongoing Department of Defense’s research interest in scaling AI to deal with large and complex military scenarios in support of wargaming for concept development, education, and analysis.
Keywords
AGENT-BASED SIMULATION, AI, BEHAVIOR MODELING, MACHINE LEARNING
Additional Keywords
reinforcement learning, wargaming
