As computer-based instruction evolves to support more adaptive training, it is becoming increasingly more evident that such programs be designed around an individual trainee's characteristics, rather than focusing just on task performance. In other words, a trainee's state (e.g. how they learn, their affect and motivation) is an important factor in performance and retention. To optimize individual performance in computer-based training Intelligent Tutoring System (ITS) technologies (tools and methods) are combining artificial intelligence (AI) knowledge representations and programming techniques with the intent to deliver instructional content and support tailored to the individual (Conati & Manske, 2009). From a holistic perspective, such tools and methods personalize training by considering an individual's historical data, real-time behavior, and cognitive measures to predicting comprehension levels and affective states (i.e. frustration, boredom, excitement). This historical and real-time interpretation of the trainee is used for concurrent adaptation of pedagogical and feedback strategies within training content.
Several ITS studies within academic settings report significant learning gains among students receiving adaptive ITS support when compared to students in a traditional schoolhouse environment (Koedinger, Anderson, Hadley & Mark, 1997; Kulik & Kulik, 1991). However, the majority of those systems supported domains with well-defined problems that require well-defined solutions (i.e. physics, algebra). With recent trends in virtual scenario-based training in the defense and medical communities, there has been a major push to simulate more ill-defined tasks that require critical decision-making and swift problem-solving. Primary issues associated with ill-defined scenario training are the lack of a suitable design framework, and determining an appropriate level of support/direction through pedagogy and feedback. This paper will compare ITS pedagogical design considerations between well-defined and ill-defined tasks, identify the variables of interest that have the greatest impact on performance and skill acquisition, and present a high-level design architecture.