Desktop simulations and digital game-based technologies have earned much attention for their potential as training interventions. Supporters view the physical realism and interactivity of the technologies as a powerful means of fostering the development of cognitive skills. The USMC is leveraging aspects of digital-game based methods and inserting them into tactical decision-making simulations (TDSs) that can supplement existing training. This paper presents a pilot study conducted to examine the training utility of TDSs. One TDS, Close Combat Marine, was assessed against traditional paper-based Tactical Decision Games (TDGs). The objectives of the study were to examine the utility of metrics for assessing improvement in cognitive skills as a result of TDSs and TDGs, and also to generate initial hypotheses regarding optimal ways of designing and implementing TDSs. Eight separate metrics were employed, including final exam scores, knowledge tests, surveys, cognitive assessments, and behaviorally anchored rating scales that assess mental models for tactical thinking. Results indicate that students in the TDS intervention were more motivated to engage in the training sessions than those in the TDG group. The findings are consistent with prior research indicating higher levels of learner motivation associated with digital game-based technologies. The results also indicate that TDSs and TDGs may be differentially beneficial. In this pilot study, for example, the TDG intervention seemed better at training mental simulation, planning, and mission focus. The TDS intervention seemed to be stronger at addressing timing considerations, execution knowledge, and team coordination. Further research examining a broader range of TDSs should focus on more accurately gauging their relative advantages and disadvantages and on guiding their incorporation into existing training programs. This research was funded by the Office of Naval Research under contract #N61339-99-D-0012 to CHI Systems, Inc. under the USMC Program Manager, Training Systems Science and Technology Division..