A boom in AI technology and presence has made AI virtually omnipresent across domains. However, an important aspect of AI adoption is the level of trust and perceived competency of the system by the human (Hancock et al., 2011). When done seamlessly, such as Google’s search algorithm or Netflix’s “Trending Now” feature, humans are provided with results that are germane to their needs, historical interests, and are more naturalistic in interaction; ultimately increasing their perception of the systems competence (Low et al., 2021). Therefore, it is of high importance to ensure that systems are designed in a way that promotes users’ trust while providing them with the support that they need as we look to integrate decision-support AI into intelligence, mission planning, and JADC2 applications. This paper presents the design of a novel system intersecting human factors, cognitive modeling, and recommendation AI to explore approaches for collaborative human-AI teaming. Under this effort, a web-based decision support AI provided recommendations for publicly available articles to answer an intelligence analysis priority intelligence requirement (PIR). We conducted a series of usability and system design evaluations that explored (a) information that users consider when making trust judgments, (b) unobtrusive behavioral measures that integrate into cognitive models to predict when trust falls, and (c) trust calibrations when cognitive model predictions did not match user actions—providing the AI an opportunity to build trust by intervening at the right time in the right way. User behavior, impressions, and self-report responses were examined to understand what user behaviors emerge when users perceive a tool to be working collaboratively. Specific guidance on designing recommendation AI that can leverage behaviors and cognitive modeling for naturalistic interaction as well as system calibration techniques to improve a user’s perception system competency are discussed.
Keywords
AI, COGNITIVE, COLLABORATIVE, DECISION
Additional Keywords