Drones are changing the face of warfighting more than any other technology in the recent past. They keep getting smaller, cheaper, and easier to deploy, however, as more and more drones are deployed, analyzing, and integrating their data is becoming a major problem. Turning a screen of 20 or more video feeds into an operational picture and creating situational awareness is an incredibly difficult task for commanders to be effective leaders.

The goal of this work is to replace the wall of video feeds with an integrated Digital Twin (DT) of the real world that is being observed by drones. A 3D model is created through LIDAR scans or photogrammetric reconstruction. This 3D model is updated in real-time, based on the video feeds from the drones. We explore different levels of detail for the updates, based only on video and/or 3D reconstruction.

As we are updating the visual representation programmatically, we can also use it to solve the second major problem of drone swarms: control. Drones usually require a pilot per drone, which doesn’t scale to large swarms. In our case, we can keep track of when any part of the Digital Twin was last updated, and after finding the least up-to-date part, we can task a drone to update it, and therefore automatically keep the Digital Twin as up-to-date as possible. This allows control of an arbitrary number of drones without manual intervention.

We demonstrate the feasibility of these approaches in a simulated environment using virtual drones observing an animated virtual world. The results show the potential and power of our proposed methods.

This approach has the potential to significantly improve the safety, efficiency, and reliability of physical systems and applications in military operations, disaster recovery, search and rescue operations, safety and emergency training, and other areas.

Keywords
REAL-TIME

Additional Keywords
Drones, Virtual Environment, Command and Control, Digital Twin