The military trains a large pool of personnel skilled in maintaining and repairing a variety of complex equipment. The U.S. Army itself requires personnel for more than 130 different Military Operational Skills. Often these trained personnel are not available for repair and maintenance of critical low density equipment in deployed locations. Augmented Reality and Virtual Personal Assistance are technologies that can supplement live training to address the challenge of affordably training personnel. In this paper, we present the system design, hardware, algorithms and initial field results for a prototype training system AR-Mentor. The system is designed to act as a personal mentor to a user, providing human-like understanding and guidance. It provides a Heads-up and Hands-free experience. The user can train anywhere and also use the system for providing guidance during actual maintenance of the equipment. The experimental system consists of a compact computer, head worn cameras, microphone, ear-buds and eyewear. Virtual Personal Assistant technology is used to provide a real-time dialog and reasoning system that supports human-like interaction using spoken natural language. The reasoning system aims to recognize the user’s intent and provides feedback to the user. The feedback and interaction occurs both verbally and by engaging the Augmented Reality system to display icons and instructions visually on the user’s eye-glasses. The inserted visual objects appear as part of the live scene and are precisely aligned to the equipment. A formative evaluation indicated that the AR-Mentor system permitted individual learners to focus on their learning needs and reduced the perceived mental demand of learning the procedure. Checks into understanding showed no difference between learning with the AR-Mentor system, as compared to learning from an instructor, or a technical manual. The evaluation also indicated the need for alternative ways to design the AR-Mentor representations around complex procedural steps.
Augmented Reality Virtual Personal Assistant for Training, Maintenance, and Repair
6 Views