Multi-agent reinforcement learning (MARL) is increasingly ubiquitous in training dynamic and adaptive synthetic characters for interactive simulations on geo-specific terrains. Frameworks such as Unity's ML-Agents help to make such reinforcement learning experiments more accessible to the simulation community. Military training simulations also benefit from advances in MARL, but they have immense computational requirements due to their complex, continuous, stochastic, partially observable, non-stationary, and doctrine-based nature. Furthermore, these simulations require geo-specific terrains, further exacerbating the computational resources problem. In our research, we leverage Unity's waypoints to generate multi-layered representation abstractions of the geo-specific terrains to scale up reinforcement learning while still allowing the transfer of learned policies between different representations. Our early exploratory results indicate efficiency in reinforcement learning and improved performance with waypoint-based navigation, pointing out the potential of waypoint-based navigation for reducing the computational costs of developing and training MARL models on geo-specific terrains. This paper discusses our framework for generating waypoint-based representations of geo-specific terrain automatically. We then present how such waypoint-based representations perform in MARL experiments with various proof-of-concept military training scenarios.

Keywords
AGENT-BASED SIMULATION;MACHINE LEARNING;TERRAIN

Additional Keywords
Reinforcement Learning