Service members’ skill and knowledge gaps can result in significant safety and readiness gaps. Recent advancements in Large Language Models and Multimodal Models can be leveraged to enhance the effectiveness of ongoing readiness initiatives such as LVC and RRL. Detailed, multimodal content (e.g. text, video, mixed reality) can be rapidly tagged and organized for applications including recommendation engines and natural language interaction with content.
For example, during an after-action review following an LVC exercise, the exercise leader can be recommended and jump to examples of a missed maneuver being executed correctly in similar contexts, delivering teachable moments to Sailors. Further, Sailors can independently access specific sections of content on demand to support performance. In both use cases, the crucial information gets to the right Sailor in the right modality to improve performance. Text-only technical publications that support repair and maintenance can limit understanding and do not necessarily connect with younger generations. Multimedia tutorials are generally more effective materials than text for instructing hands-on procedures; however, keeping multimedia content up-to-date has been labor intensive as frequent updates to official technical publications occur.
We outline an automated, AI-enabled schematic linkage between a given task, e.g. inspecting the nose of an F/A-18, and its related content objects in various media formats. Users can toggle between video and text instructions while executing a complex procedure to ensure the necessary detail is delivered to the user, avoiding costly errors. Automated linkage of an official procedure step and its aligned content objects also enables flagging a section of content for update or human review if the official procedure step is changed, ensuring that content remains up-to-date. In tests that leverage car maintenance procedures and tutorial videos, we demonstrated the ability to automatically tag and maintain content using AI methods.