Main Subcommittee Category: Simulation

How would you label your submission?: Academia

Abstract:

In the domain of the U.S. Army modeling and simulation (M&S), the availability of high-quality annotated 3D data is pivotal to create virtual environments for training and simulations. Traditional methodologies for 3D semantic/instance segmentation, such as KpConv, RandLA, Mask3D, etc. are designed to train on extensive labeled datasets to obtain satisfactory performance in practical tasks. This requirement presents a significant challenge, given the inherent scarcity of manually annotated 3D datasets, particularly for the military use cases. Recognizing this gap, our previous research leverages the One World Terrain (OWT) data repository’s manually annotated databases, as showcased at I/ITSEC 2019 and 2021, to enrich the training dataset for deep learning (DL) models. However, collecting and annotating large-scale 3D data for specific tasks remains costly and inefficient.

To this end, the objective of this research is to design and develop a comprehensive and efficient framework for 3D segmentation tasks to assist in 3D data annotation. This framework integrates Grounding DINO (GDINO) and Segment-anything Model (SAM), augmented by an enhancement in 2D image rendering via 3D mesh and Gaussian Splatting. Furthermore, the authors have also developed a user-friendly interface (UI) that facilitates the 3D annotation process, offering intuitive visualizations of rendered images and the 3D points. To evaluate the proposed annotation framework, outdoor scenes from STPLS3D (Chen, et al., 2022) and indoor scenes collected using Matterport were used to conduct comparative experiments between manual methods and the proposed framework, focusing on 3D segmentation efficiency and accuracy. The findings from these experiments demonstrate that our proposed framework surpasses manual methods in efficiency, enabling faster 3D annotation tasks without compromising on accuracy. This indicates that the potential of the framework to streamline the annotation process, thereby facilitating the training of more advanced models capable of understanding complex 3D environments with enhanced precision.

Keywords: 3D;3D SCANNING;MODELING

Additional Keywords not in the list above: Open Vocabulary Object Detection, 3D point cloud segmentation, 3D point cloud annotation

Menu

Zifan Yu