Making quality decisions under stress is a critical Warfighter attribute. Enhancing decision making skills requires two elements: (1) assessing decision making quality and (2) facilitating effective decision making training. Current methods for assessing decision making in real-life settings are based on retrospective process tracing; however, this method often yields unreliable data due to memory distortion, biased interpretation, and inability to recall facts that were not encoded in long-term memory (Reidi, Brandstatter & Roithmayr, 2008). Current training modes typically focus on covering gaps in states of knowledge rather than on the more appropriate cognitive skills (Klein & Baxter, 2009). Thus, developing effective training for decision making is hampered by a number of limitations.

Ideally, assessments of decision making quality would be made in real-time and in situ. Training would be done similarly but using computer automated After-Action Review programs that capture and then replay the decision process in situ for facilitating the desired cognitive changes.

The purpose of this investigation was to assess the effectiveness of a simulation engine created to test decision-making skills in naturalistic, virtual environments. For this experiment, novice (n=23) and veteran (n=39) firefighters were exposed to two stressful "real-world" virtual simulations (Difficult Tradeoffs, High Time Pressure) while decision-making strategies, physiological responses, and situation awareness via cue recognition were assessed. The results suggest that experience does not immunize from making suboptimal decisions. Analysis of the distribution of decision making strategies suggests that the Recognition-Primed Decision model (Klein, 1998) is employed often when Time Pressure is the stressor; it is not as prevalent when the stressor is Tradeoffs. Physiological responses suggest that veteran firefighters have greater autonomic arousal than novices under similar situations. Post-simulation feedback indicates a high level of immersion and training usefulness.

These results support the effectiveness of the developed simulation engine to assess decision making quality in real-time and in situ albeit using virtual reality. Therefore, this framework has promise for use in warfighting.