Keywords: BEST PRACTICES;MEASURES;METRICS;ROI
Additional Keywords not in the list above: COST, BENEFIT, VALUE
Learning Objective 1: Explain what M&S ROI is and what it is not
Learning Objective 2: Summarize the differences in calculating M&S ROI where profit is not the motive - e.g., DoD
Learning Objective 3: Construct metrics for use in the ROI calculation
Learning Objective 4: Recall real world examples and considerations (level and type of ROI metrics)
Learning Objective 5: Describe how to apply the ROI concepts provided to your / real world M&S use cases
Abstract:
When requirements increase faster than available resources, decisions on how to allocate the resources among various programs and projects are required. Project and program managers must show why their systems are worthy of continuing and how they are adding deterrance through integrated training and readiness. While many factors influence these important decisions, return on investment (ROI) should play a key role. ROI has been an essential factor for many years, however, lately we have seen many instances where the term is used incorrectly. For example, “The use of System X saved 23% of classroom hours,” is not ROI. Furthermore, traditional ROI requires a return or revenue stream to calculate the benefit of the investment. However, in military applications there is most likely not a revenue stream. How does one then calculate the return?
This tutorial builds on a study performed for the Modeling and Simulation Coordination Office in 2009 and published in the Acquisition Review Journal in 2011. In the tutorial, attendees will be presented the definition of ROI, some examples of ROI and some examples of the incorrect use of ROI. Then some of the unique challenges to the DoD relative to the M&S environment will be presented with the proposed solutions from the study discussed. In that discussion, how to construct solid usable metrics for use in the ROI calculation for M&S will be presented with real-world examples given. Additionally, an examination of how ROI can differ depending upon your point of view (management, program, or system level) and what things are considered in the calculation. Finally, examples that put all the ideas together and show ROI in some different scenarios will be presented and discussed. The end-goal is that each attendee will come away with an understanding of how best effect ROI use as well as how it can be misapplied; how to calculate it and the unique challenges that arise when there is no revenue stream; how to overcome those challenges and develop measurable metrics for use in the ROI calculation; and finally, how all of the principles come together in detailed examples and how the appearance of ROI may differ depending upon your management level. Using these methods, attendees will walk away with being better able to defend their M&S programs and projects against the ever-present funding axe.