A large body of research indicates that authentic problem-solving experiences are crucial in achieving mastery of complex subject matter. It is relatively rare, however, that problem-solving environments for serious applications are instrumented in a way that makes it possible to record and automatically categorize every meaningful action in the problem-solving context. We have developed such a tool and have begun to use it to study complex learning and problem solving.

The ASW Sandbox, a tool for learning about anti-submarine warfare (ASW) in the context of tactical planning was developed for a course taught at the U.S. Navy Surface Warfare Officers School (SWOS). The tool offers two modes: an Instructor Mode for rapidly developing tactical scenarios and problems, and a simpler Problem Mode for delivering interactive scenarios and recording learner actions. Instructors and students have chosen to utilize the tool in five ways. (1) Instructors develop simple scenarios to illustrate particular tactics in ideal contexts. They then record their own solutions to these problems. In class, they play back a recorded solution while describing processes and procedures and explaining the reasons for actions and effects. (2) Instructors develop more complex and/or realistic scenarios that test different types of tactical knowledge. Students are divided into groups and given these problems in class. Their recorded actions are played back through the simulator as they debrief the other members of the class. (3) On an experimental as-needed basis, ASW problems are used to evaluate students, when conventional paper-and-pencil tests are suspected of not reflecting actual competence at the task. (4) Some individual students voluntarily work with additional instructor-authored problems to expand their understanding of ASW. (5) Some students also utilize a version of Instructor Mode to author new problems, so that they can test the limits of tactics.