Until recently, most existing synthetic character behavioral models for military training simulations were either rule-based or reactive with minimal built-in intelligence. As a result, such models could not adapt to the characters’ experiences, be they with other synthetic characters, the environment, or human trainees. Multi-agent Reinforcement Learning (MARL) models multiple agents that learn by dynamically interacting with an environment and each other, presenting opportunities to train adaptive models for both friendly and opposing forces to improve the quality of synthetic characters. Still, military environments present significant challenges since they can be stochastic, partially observable, nonstationary, and doctrine-based. This paper introduces a scout mission scenario modeled within the Rapid Integration and Development Environment (RIDE) on a geo-specific terrain designed to leverage deep learning and simulation-generated experiences in a MARL framework and presents results from exploratory experiments. Furthermore, it discusses the trade-offs between various design choices, including discrete versus continuous observation spaces and bootstrapping behavior policies with pre-trained models.
Keywords
AGENT-BASED SIMULATION,BEHAVIOR MODELING,DEEP LEARNING
Additional Keywords
multi-agent reinforcement learning