Augmented reality is a key technology which promises to change the future of warfare. Hybrid AR military training will enable outdoor engagements with long distance enemies, both real and simulated. For live military training over a large area, a 3D model of the world must be maintained based on sensor observations. To proliferate AR technology, 3D reconstruction algorithms must utilize the low cost and pervasiveness of video camera sensors, from both overhead and soldier-level perspectives. Mapping speed and 3D quality must be balanced to enable live hybrid training in dynamic environments.
Given these requirements, we present a live 3D reconstruction pipeline for large scale mapping for military applications given only live video. Our method, ORB-Recon, uses ORB (Oriented FAST and Rotated BRIEF) feature points to reconstruct a 3D model of the world in real-time, while tracking the location of dynamic objects such as humans and vehicles. ORB-Recon is derived from and published open source and free to use by all. To evaluate the challenges of 3D reconstruction quantitatively, we utilize the autonomous driving academic benchmarks KITTI and KITTI-360, minimizing the error in point cloud 3D placement compared to these baselines. We measure 3D reconstruction performance to common structure from motion, visual-SLAM, and photogrammetry techniques. ORB-Recon outperforms other state of the art techniques when combining speed and reconstruction quality metrics. We qualitatively stress-test against two new benchmarks: 1. Hiking the Grand Canyon while updating a 3D model at both a local and massive scale; 2. Military-simulation paintball from multiple live perspectives. Finally, we investigate pulling 3D terrain elevation maps from One World Terrain, as well as pushing back live updates from soldier-level perspectives.
Keywords
3D,3D SCANNING,3D-STEREO PROJECTION,AUGMENTED AND VIRTUAL REALITY (AR/VR),GEOSPATIAL DATA ,MIXED REALITY,RAPID MODELING,REAL-TIME,SERIOUS GAMES,SIMULATIONS,SYNTHETIC ENVIRONMENT,TACTICAL,TERRAIN
Additional Keywords
3D Reconstruction, Visual SLAM, Structure from Motion