Assessing the effectiveness of a plan, given multiple potential scenarios, is a common problem for analysts, especially in the military domain. This problem can seriously impact the safety of the people that are involved in planned missions. More precisely, the availability of multiple models, with varying levels of fidelity, leads to the complex task of selecting the best model(s) to assess the effectiveness of a plan. Under time constraints, optimal model selection depends not only on the fidelity of the models at hand, but also on the nature of the possible scenarios the plan applies to, such as the potential presence of stochastic variables and the number of different scenarios that have to be evaluated in order to obtain a reliable estimate of the true effectiveness of the plan. In this paper, two algorithms are presented to maximize the reliability of the obtained plan effectiveness under time constraints. To this end, the algorithms select the best model(s) as well as the most appropriate scenarios. Both algorithms have been tested on syn-thetic data as well as on two Navy-related use cases. Results show that both algorithms reach a higher level of reliability within the given amount of time than conventional approaches. Thus, they allow analysts to better assess the effectiveness of their plans and therefore they increase the safety of everyone involved in planned missions.