Developing and deploying complex systems can be highly challenging, particularly with the issue of un-intended or emergent behaviours. Existing test methods are well adapted to low level test, demonstrating the performance of components in their un-integrated state, or high level demonstrations of integrated systems, but which lack the depth of test to find and fix emergent behaviours, particularly where those may be introduced by influences outside the system of interest. This is particularly important where the safe operation of the system matters to those who mission matters.

One such complex system is autonomous vehicles. This paper will seek to discuss the issues of identifying the impact of emergent behaviours, and approaches to verifying safe performance of complex systems in the real world, by testing in the synthetic environment. This paper will build upon the findings of the OmniCAV programme in verifying the safe behaviour of an autonomous vehicle and draw parallels with other complex systems used in defence.

OmniCAV is a consortium project that is partly funded by the UK Government. It aims to deliver a highly realistic simulation environment for AV stack verification that considers all road users and road types. The programme has developed a unique synthetic environment integrating a high fidelity survey of urban and rural Oxfordshire, with other simulated road users allowing all conditions to be simulated and evaluated. This is being delivered in combination with real-world testing to support the assurance of the safety case for deploying a self-driving car. As well as XPI’s work on OmniCAV, potential approaches for integrating multi-part simulators for generating coherent and rich digital twins associated with such autonomous systems testing will be addressed.

In concluding, some of the key technical challenges that remain in this domain will be identified, including interpretation of results and assuring completeness of testing.