Large-scale, distributed live, virtual, and constructive training exercises show much promise in providing effective training, but many challenges must be addressed. By their very nature, these exercises involve individuals who have different experience levels, who perform different jobs or roles, and who necessarily have different training requirements. For any two participants in a common training event, individual proficiency will influence their ability to perform tasks during the event. Interactions or interdependent tasking can result in overwhelming some participants while boring others. As we enhance simulator-based training capabilities with real-time performance measurement and the engineering of adaptable content, there is a clear opportunity to use these capabilities to better serve the trainee. This paper details an approach to leveraging measurement theory and scenario engineering technologies to produce a tool which will inform the "threading together" of scenario experience items based on a diverse set of targeted training objectives.
Specifically, Item Response Theory (IRT), a testing methodology that uses mathematical models to select appropriate test items for individuals based on observed levels of proficiency, can be effectively adapted to simulator based training. Through the selection of appropriate "scenario experiences," or items, IRT can facilitate an adaptive training environment where disparately skilled individuals can train collectively and benefit from training tailored to their skill level. We present trainees with a seamless pre-test, adaptive scenario, and post-test training experience. This procedure develops initial trainee proficiency estimates, provides developmental content based on individual capabilities, and results in an assessment of each trainee's progress. Collective training objectives are met through the presentation of individually tailored content relevant to the targeted knowledge, skills, and abilities. What results is a seamless scenario experience for disparately skilled trainees participating in a team or large scale exercise.