The Advanced Distributed Learning (ADL) vision encompasses the use of scenario-based simulations to provide a rich environment for training complex tasks. At the same time, it introduces a complex assessment environment, which creates challenges in the accurate and efficient diagnosis of student needs as frequently student behaviors can be interpreted in several ways. Diagnosing student learning needs, consequently, becomes problematic. Unfortunately, there are currently no best practice guidelines for extracting and making use of performance data from a simulation-based training environment. However, methods that address these challenges are required for the successful integration of simulation-based training into the ADL Initiative.

The research described in this paper is investigating the development of a scenario-based performance assessment method that leverages the Shareable Content Object Reference Model (SCORM), while using information on trends to isolate individual learning needs. Specifically, SCORM 2004 specifications enable a single Shareable Content Object (SCO) to be linked to (i.e., to set a value of or the status of) multiple learning objectives. Although the potential impact of this capability for assessment has not been widely recognized to date, it provides a means to interpret relatively complex responses in scenario-based training in terms of all of the learning objectives that may be implicated by a given action. The methods developed under this project will support changing the measures that reflect a student's mastery of the underlying learning objectives as a result of study, practice, and forgetting. Further, hypothesis-testing methods will be employed to resolve ambiguous diagnoses of learner needs.

These methods are being applied to the development of an ADL prototype in the context of Marine Air Ground Task Force (MAGTF) command and control training. The MAGTF XXI Tactical Decision-Making Simulation (TDS) is being employed as the simulation-based training environment. The proposed paper will describe this research and its application to the development of a simulation-based training application for the MAGTF commander.