How can we build AI that robustly recognizes how well a team is doing from behavioral data that exhibit the full range of human complexity and dynamics? One method is cognitive inversion. An AI with a causal model of human behavior that is sufficiently dynamic to account for behavioral variability and teammate interactivity and scoped to a set of tasks and interactions of interest, combined with probabilistic program inference, can invert that behavioral model to generate hypothesizes about the underlying goals and causes of observed behavior. As a tutor, coach, or teammate, the AI can then intervene to assist when needed. Here, we describe a prototype prescient, socially intelligent coach (PSI-Coach), a system to perform cognitive inversion, and supporting components. PSI-Coach monitors team members to: recognize their goals, mental states, and behaviors from dynamic streams of actions by combining probabilistic programming inference with a cognitive architecture designed to capture human variation; use those recognized cognitive states to infer team shared mental models and whether they are in alignment or skewed; analyze these goals, mental states, behaviors, and shared mental models to compute practical, real-time team performance indicators; and use all of this information with interactive-narrative technology to plan minimally-intrusive, effective, strategically timed interventions that help to improve team performance. In experiments, we demonstrated PSI-Coach ability to automatically identify team process problems unique to different teams and their situation dynamics, and to provide timely, tailored intervention content that improved team processes within 60 seconds. PSI-Coach showed a 35% increase over two comparison systems’ real-time inferences (p<0.05) and a 42% and 68% increases in agreement with human coaches on interventions over a baseline inference method (p<0.05). Compared to no-advisor teams, PSI-Coach showed a 60% improvement in aligned-team-priorities (p=0.09) and 13% improvement in coordinated-communication (p=0.09) compared to baseline trials with no interventions.
Keywords
AI, BEHAVIOR MODELING, COGNITIVE
Additional Keywords
artificial intelligence, probabilistic programming, cognitive inversion, human behavior modeling, human-machine teaming