The goal of training is to establish or increase knowledge and performance of skills, with improved performance realized in an operational setting. But quantifying the impact of training in operational terms is oftentimes seen as unachievable. Stakeholders are left to make acqusition decisions based on requirements met, not on how much of an impact a given training solution will have on operations.

By starting with integrating clear measures of operational impact right at the beginning of an agile product development lifecycle, insightful supporting and transfer documentation can build knowledge and skills based on clear objectives that directly leverage those measures of impact. This can then be assessed in an incremental approach, and the documents become readily adaptable to formal training requirements.

Implementing the key steps, one can best quantify the learning impact on the individual, team and organization. (1) Clearly define the identified performance gap in terms of operational impact; (2) develop impact-based learning objectives to address the gap, and (3) establish clear metrics to measure achievement of learning objectives and anticipated performance outcomes. To be successful, evaluating oeprational impact requires a transparent upfront needs analysis.

Using Kirkpatrick's model of training evlauation can help to ensure operational impact is evaluated across all four evaluation levels: Reaction (Was the training well received?); Learning (Did the trainees learn?); Behavior (Did this learning result in changed behaviors/transfer of training?); Impact (Did the training make the desired organizational impact?). Through this Operational Impact Analysis, one can align business indicators with skills/knowledge gained, and provide quantitative validation that trianing will have the desired impact. Stakeholders want to see impact in terms of time, lives, or money saved. Incorporating user-in-the-loop evaluations implementing key metrics of success during early product releases can provide operational impact indicators, and not only show the potential value of the training solution, but also guide development in identifying opportunities for increased training transfer capabilities, often in a more compressed timeline to sustainment.

This tutorial will provide attendees with insights on why Operational Impact is critical for training success, and how measuring Operational Impact can be integrated into the training development process. Implementing methods of evaluation can provide attendees with a means to formulate outcomes that will more clealry demonstrate the value and impact of their training solution - not only with initial knowledge and skill transfer, but also the overarching beneficial impact to the program office and organization.

Amy Taber

Affiliation

Refine Results

Operational Impact: Quantifying Training Solution Value