Behaviors of the synthetic characters in current military simulations are limited since they are mostly generated by rule-based and reactive computational models with minimal intelligence. Such computational models cannot adapt to reflect the experience of the characters, resulting in brittle intelligence for even the most effective behavior models devised via costly and labor-intensive processes. Observation-based behavior model adaptation that leverages machine learning and the experience of synthetic entities in combination with appropriate prior knowledge can address the issues in the existing computational behavior models to create a better training experience in military training simulations. In this paper, we introduce a framework that aims to create autonomous synthetic characters that can perform coherent sequences of believable behavior while being aware of human trainees and their needs within a training simulation. This framework brings together three mutually complementary components. The first component is a Unity-based simulation environment - the Training Simulation Software (TSS) API - supporting One World Terrain (OWT) models and capable of running and supporting machine learning experiments. The second is a novel multi-agent deep reinforcement learning library that can interface with a variety of simulation environments, and that can additionally utilize a variety of reinforcement learning algorithms. The final component is the Sigma Cognitive Architecture that will augment the behavior models with symbolic and probabilistic reasoning capabilities. We have successfully created proof-of-concept behavior models leveraging this framework on realistic terrain as an essential step towards bringing machine learning into military simulations: (1) in order to improve the quality and complexity of non-player characters in training simulations; (2) in order to create more realistic and challenging training experiences while reducing the cost and time to develop them; and (3) in order to make simulations less dependent on the availability of human participants.