Although similar, learning and performance are not synonymous. Specifically, performance is defined as the quality, rate, or accuracy of a specific behavioral response at a specific point in time. By comparison, learning is defined as a relatively permanent change in knowledge, skills, or understanding across tasks, across time, and/or across environments (Christina & Bjork, 1991). While this distinction is applicable to all training methods, it is particularly relevant in the context of simulation-based training (SBT), because instructors need to ensure that learners have mastered the critical work-related skills, rather than simply having learned “how to game the simulator.� In a recent I/ITSEC paper (Beaubien, Stacy, Wiggins, Keeney, Walwanis et al., 2015), measures of learning (MOL) were conceptually and empirically differentiated from measures of performance (MOP) during simulation-based carrier landing practice. The MOLs and MOPs provided vastly different interpretations of the data. In particular, it was shown that a singular reliance on MOPs would have provided the Navy with incorrect guidance regarding the design of future flight simulators for training carrier landing skills. This follow-on paper explores nine theoretically-derived methods for measuring learning. Several are rooted in the dual-process theory of decision-making (Evans & Stanovich, 2013, Kahneman, 2011), which postulates two brain processes – an analytical one which is characteristic of novices, and an intuitive one which is characteristic of experts – that operate in parallel. The remaining methods are rooted in a generalized model of skill acquisition (Dreyfus & Dreyfus, 1980), which postulates that there are predictable patterns of cognitive and behavioral development over time as one becomes more expert-like. Each assessment method is described using lay terminology so that non-scientists can integrate them into their own training efforts, SBT or otherwise. Specific examples from the published literature are also used to illustrate key points.
How Can We Measure Learning? Let’s Count the Ways!
4 Views