This paper describes research directed at determining the validity of measures of cognitive readiness, the mental preparation needed to be competent in the performance of complex tasks in a military environment. The Surface Warfare Officers School (SWOS) Multi-Mission Team Trainer (MMTT) is used to assess the performance of Tactical Action Officers (TAOs) operating in a simulated Combat Information Center (CIC). Scenarios require the TAO to defend against air, surface, and subsurface threats. A computer-based assessment system was developed for gathering data, analyzing, and reporting results. The system supports the assessor in rating the quality of learner responses to various scenario events on an "optimal," "adequate," and "other" rating scale, providing prompts for behaviors to record and questions to ask. The point and click interface minimizes interference with the assessor's observation of events and performance. The system automatically records and scores performance. Measures include (a) observed actions, e.g., reports appropriately communicated; (b) responses to mid- scenario probe questions, e.g., expectations regarding a track; (c) part-task anticipation requiring the learner to respond to short scenarios presenting a situation, e.g., identification of the greatest threat; and (d) critical events presenting cognitive "traps" designed to expose a cognitive error. Measures are mapped to cognitive constructs including situation awareness, decision making, communication, problem solving (formulating tactics plans), command and control (implementing and monitoring tactics plans) and acting effectively in a timely manner. The system aggregates results and generates graphs of performance by construct, identifies areas of strength, and provides recommendations for improvement. The paper describes the tool, the results of preliminary testing, the strengths and weaknesses of the approach to assessing cognitive readiness, and plans for future research.