The military and security forces maintain multiple MOUT sites to train personnel for dangerous urban operations. Some of these facilities are instrumented for automatic tracking and recording of an individual trainee's actions; this is done to adapt the training conditions in real time and to provide detailed information for after action reviews. Today, tracking capabilities are quite limited, and generally include video cameras installed across the facility and GPS systems for outdoor tracking. No cost effective systems exist that are capable of tracking the location, pose, and gaze direction of individual trainee and the location and pose of their weapons both indoors and outdoors. There is a need for systems that can provide such measurements over wide areas, such as MOUT sites that cover multiple square miles and include numerous buildings.

In this paper we present a system for tracking the trainee's location, head orientation, and weapon orientation that provides high precision and does not require an instrumented site. Tracking is achieved only with sensors mounted on the individual trainees. These sensors include helmet-mounted video cameras and an inertial measurement unit. The vision system estimates both relative motion based on visual odometry and absolute position and orientation based on landmark matching. The 3D landmark database is built autonomously prior to the exercise. The system seamlessly handles transitions into and out of GPS-denied environments (buildings, dense forests) by maintaining pose relative to what the cameras are seeing in addition to GPS. We have demonstrated the viability of this technology in urban, rural, desert, forest and indoor environments on human-wearable platforms.