Team training has been identified as critical to the operations of the Department of Defense (DoD) due to the complex and frequent interactions required in military teams. Effective training is necessary to develop complete understandings of the task and to build cooperative teams. Intelligent Team Tutoring Systems (ITTSs) have the potential to reduce training costs, improve learning, and increase feedback consistency in comparison to traditional human tutors. Currently, ITTSs are underdeveloped due to the state of the technology and the complex nature of intelligent agents, which require a variety of considerations and many hours to create. In this paper, the authors explore the impact of automated tutor feedback and team composition on performance for participants tasked with identifying and tracking enemy combatants.
Thirty-seven three-person teams, each composed of two spotters and one sniper, were tutored on their surveillance task performance over four trials. The scenario was constructed using Virtual Battle Space 2.0 (VBS2) and a version of the Generalized Intelligent Framework for Tutoring (GIFT), which assessed learners and delivered real-time feedback. In 18 teams, members received private, individualized feedback, while in 19 teams, members received individualized public feedback (i.e., their teammates could observe). Additionally, all teams experienced a change in team composition as the sniper and one of the surveillance spotters traded roles for the fourth trial. Each team’s performance in the task was assessed. Evidence of training effectiveness is observed in participants’ subjective performance and task workload. While feedback privacy was not found to influence the subjective performance, an effect was found for objective performance. These results about the effectiveness of feedback in team settings will influence the future study and development of ITTSs for the military by adding to the literature on how automated feedback should be designed within team training settings.