Abstract:

Due to rapid technological evolution and dynamic operational challenges, the ability to train military personnel effectively and quickly is a strategic necessity. However, a single change in Army doctrine requires changes across many courses and training resources, with some estimates indicating a backlog of several hundred man-years of updates. The AI-Assisted Revision of Content (ARC) tool uses AI to analyze doctrine and training documents (lessons, slides), to suggest which sections of training need to be updated. ARC was designed with three main capabilities: 1) Ingestion: Specialized scraping of documents ingested (Field Manuals, Army Publishing Directorate doctrine, lesson plans, presentations), which extracts passages, page references, and section structure, where possible; 2) Normalized Hybrid Search: Analyzing document-to-document alignment and similarity comparisons, using hybrid search with both sentence transformers and keyword embeddings (Okapi BM25), with a novel best-match normalization technique; 3) Change Analysis: Specialized user interfaces to enable comparing training documents, both in pairs (e.g., previous vs. latest doctrine) and triads (e.g., a slide deck referenced against its older vs. current doctrine source). Guided alpha testing collected formative input from seven Army training Centers of Excellence (CoE's), which determined design priorities to refine ARC capabilities. Results from hands-on beta tests with four Army Centers were highly positive, strongly agreeing that ARC would “increase productivity” (average rating 5.5 on a 6 point scale). However, CoE's reported differences in their needs, such as their typical changes (e.g., reference updates vs. explaining new concepts) or detailed auditing (e.g., safety-critical instructions from a technical manual). Ongoing research on ARC is exploring annotating and suggesting changes in editable documents that soldiers use every day (e.g., SharePoint, Word). Long term, strong demand by training centers suggests that sandboxes and rapid transition infrastructure should be implemented to support AI ecosystems for content development.

Keywords: AI;EMERGING TECHNOLOGIES;MACHINE LEARNING;MILITARY LEARNING;PEDAGOGICAL DESIGN;READINESS

Joel Matthew

Affiliation

Biography

Refine Results

Training Developer Feedback on AI for Revision of Content (ARC)