A criticism of Artificial Intelligence and Machine Learning (AI/ML) techniques is brittleness. This means that an AI/ML technique may perform well when analyzing a known data set, but performance will degrade when new data is introduced. While this degradation is apparent when new data belongs to an existing category, degradation is even more pronounced when data from a previously unseen category is introduced. Additionally, many AI/ML solutions do not have a mechanism to detect outliers or noise, instead they force these data points into categories where they do not belong. Despite these weaknesses, our global, digital world necessitates the use of these AI/ML techniques to sift through an ever-expanding data pool.
Dynamic, Unsupervised Clustering by Algorithmic Triangulation (DUCAT) does not fall victim to these shortcomings. It is designed to handle streaming data sets and adds new classification types as new data types are observed. It is not limited to a certain number of categories, and it does not need to be retrained as new data types are introduced. Additionally, it can detect noise and categorize it as such.
This paper will describe DUCAT’s process which consists of first comparing new data against known clusters, then searching for new clusters and creating cluster definitions as new clusters are found. DUCAT has been verified by segmenting a dataset of simulated radar pulses into a series of portions which are individually fed into the system. The efficacy of the system is shown by a confusion matrix comparing the DUCAT labels to truth as well as classification cohesion across portions.
This system was developed to classify radar pulses to train operators by highlighting novel signals for education or offloading classification and noise filtering work, but the system can be utilized in scenarios beyond radar classification and this paper will explore alternative applications for DUCAT.
Keywords
AI, DATA, MACHINE LEARNING
Additional Keywords
Unsupervised Learning, Clustering, Dynamic