Model validation is a technically challenging task for simulation projects. One-shot validation of conceptual models is not practical for most simulation system projects due to the inherent logical complexity of individual models, interdependencies between models, and the diverse viewpoints of the stakeholders.
Fidelity, in particular, is an issue that permeates these characteristics. The required level of fidelity is largely determined by the expectations of the users and sponsors about the simulation system, e.g. the kinds of analyses that can be accomplished. Tacit assumptions that are implicit in the stakeholder viewpoints and that underlie all modeling activities gradually become explicit through model validation activities. Moreover, harmonizing assumptions as they become explicit and clearly spelled out requires negotiations, which take multiple rounds of exchange.
Therefore, it is often difficult for stakeholders with diverse viewpoints to align on rules, boundaries, methods and constraints, and assumptions throughout the validation process. Therefore, model validation, by nature, is a multiple-stakeholder activity, that requires a specifically defined process per project, so that all aspects of the activity, varying from stakeholder management to application of technical validation methods, can be executed in gradually increasing levels of agreement and harmony.
In this paper, we argue that the aforementioned difficulties can be relieved by the establishment and application of iterative and incremental approaches for model validation activities throughout a project. Specific practices have to depend on the specifics of the project, such as model fidelity requirements, availability of field data for validation activities, accessibility of subject matter experts, etc. Nevertheless, a simulation system development project can highly benefit from a custom-defined iterative and incremental process for model validation, inspired by corresponding software development principles.
Keywords
AGENT-BASED SIMULATION, FIDELITY REQUIREMENTS, VERIFICATION, VALIDATION AND ACCREDITATION (VV&A)
Additional Keywords
iterative and incremental development, stakeholder management