Off-road autonomous vehicles (AVs) are required to operate under extensive uncertainty conditions due to the large variation in natural, unstructured environments. For such AVs, developing and validating perception models based on sophisticated artificial intelligence (AI) algorithms, is extremely challenging. A large amount of reliable annotated data from a variety of diverse environments is required, though acquiring such data can be costly and time-consuming. Off-road environments lack the simulation tool sets, datasets, and algorithms available for urban environments. These limitations can be overcome by using a digital twin for data generation, physical validation, and algorithm testing. However, creating large-scale, high-fidelity digital replicas of open terrain scenarios requires expert intervention, as well as tedious and time-consuming work.

We present a novel algorithm and implementation for the automatic and efficient generation of large-scale, high-fidelity open terrain digital twins. The algorithm relies on standard, publicly available geospatial data, such as 2D raster maps obtained from aerial or satellite images and possibly a digital terrain model (DTM). This algorithm comprises two consecutive processes: first, an AI method converts the geospatial raster map into a material map, where each pixel represents a predefined material (road, soil type, rock, vegetation type, etc.). An automatic pipeline then combines the material map with the raster and DTM to create a simulated environment that is perceptually realistic and geospecific. No specialized graphic design or programming skills are required for these processes. 

Our automated pipeline is demonstrated with Unreal Engine 5, although other simulation platforms can also be used. Our results demonstrate that high-fidelity open-terrain digital twins can be generated automatically with minimal human involvement. In rigorous field tests, we demonstrate the applicability of these digital twins for AI training and validation, AV traversability analysis, and more. Finally, we discuss limitations and future work.

Keywords
AI, DEEP LEARNING, SIMULATIONS

Additional Keywords
DIGITAL TWIN,SEMANTIC SEGMENTATION