Keywords

AI

Abstract

Generative AI is advancing rapidly, transforming human-AI (HAI) teams across training and operational environments. Yet, generative AI performance alone does not guarantee mission success, team performance, or effective HAI collaboration. These outcomes depend on effective team design using knowledge from team cognition theories, deliberate system architecture leveraging the strengths of agentic and compound AI systems, and interaction methods that support both human and AI team participants' understanding of the shared context. By combining principles from team science and innovative compound AI architecture design, we can enhance the impact of AI technologies on HAI teams, making them more adaptive, resilient, and mission-effective. This tutorial, led by an organizational scientist specializing in shared cognition and HAI teaming applications and a Chief AI architect focused on operationalizing generative AI for mission-critical systems, will lean on evidence from recent research and practical experience to offer all audience members, from AI developers to training professionals and leaders, an accessible and comprehensive framework for designing, implementing, and evaluating HAI teams for military training and beyond.

The first half will explore foundational principles of team science and their application to HAI collaboration and multi-agent AI systems. Participants will be introduced to key team cognition frameworks, including Transactive Memory Systems, Shared Mental Models, and Interactive Team Cognition, and how these concepts translate to agentic AI teams. The session will also cover state-of-the-art assessment and benchmarking methods for evaluating agentic AI and HAI team performance, highlighting challenges in trust calibration, adaptability, and decision-making in high-stakes contexts.

The second half will focus on compound AI architectures and system design that facilitate effective HAI and agentic AI teaming. We will review modern techniques such as Retrieval Augmented Generation, tool calling, and agentic architectures with advanced memory representations, summarizing how they support HAI collaboration by enabling AI systems to learn team context and priorities through naturalistic interactions. Participants will gain insights into data requirements, processing, multi-agent formulation, HAI role definition, and system structures needed to support successful HAI interactions in mission critical scenarios.

Throughout the tutorial, participants will gain a deeper understanding of how team science can inform HAI system design and how compound AI architectures that consist of large language models (LLMs), multimodal foundation models (MFMs), agents, and tools, can be orchestrated to support effective teaming. This interactive session will include case studies, discussion prompts, and Q&A opportunities, ensuring participants leave with actionable, cutting-edge insights for real-world operational settings.